-- Logs begin at Thu 2022-10-20 08:19:36 UTC, end at Thu 2022-10-20 09:17:46 UTC. -- Oct 20 08:52:04.701988 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 20 08:52:04.710941 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 20 08:52:04.712889 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 20 08:52:04 2022] *** Oct 20 08:52:04.712889 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 20 08:52:04.712889 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 20 08:52:04.712889 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: nodename: dv-cinder-master-3-iscsi Oct 20 08:52:04.712889 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: machine: x86_64 Oct 20 08:52:04.712889 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: clock source: unix Oct 20 08:52:04.712889 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: pcre jit disabled Oct 20 08:52:04.712889 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: detected number of CPU cores: 4 Oct 20 08:52:04.712889 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: current working directory: / Oct 20 08:52:04.712889 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: detected binary path: /usr/bin/uwsgi-core Oct 20 08:52:04.713635 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: your processes number limit is 47680 Oct 20 08:52:04.713635 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: your memory page size is 4096 bytes Oct 20 08:52:04.713635 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: detected max file descriptor number: 2048 Oct 20 08:52:04.713635 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: lock engine: pthread robust mutexes Oct 20 08:52:04.713635 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: thunder lock: enabled Oct 20 08:52:04.713635 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 20 08:52:04.713635 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 20 08:52:04.751691 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: Python main interpreter initialized at 0x55fde78ed9c0 Oct 20 08:52:04.751691 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: python threads support enabled Oct 20 08:52:04.751691 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: your server socket listen backlog is limited to 100 connections Oct 20 08:52:04.751691 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: your mercy for graceful operations on workers is 90 seconds Oct 20 08:52:04.752006 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: mapped 403077 bytes (393 KB) for 2 cores Oct 20 08:52:04.752089 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: *** Operational MODE: preforking *** Oct 20 08:52:04.752339 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: *** uWSGI is running in multiple interpreter mode *** Oct 20 08:52:04.752339 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: spawned uWSGI master process (pid: 103835) Oct 20 08:52:04.753089 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 20 08:52:04.754278 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: spawned uWSGI worker 1 (pid: 103836, cores: 1) Oct 20 08:52:04.754278 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: spawned uWSGI worker 2 (pid: 103837, cores: 1) Oct 20 08:52:04.754278 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 20 08:52:05.413230 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 20 08:52:05.413817 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 20 08:52:06.494901 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 08:52:06.494901 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: from cryptography.utils import int_from_bytes Oct 20 08:52:06.495551 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 08:52:06.495551 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: from cryptography.utils import int_from_bytes Oct 20 08:52:06.495673 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 08:52:06.495673 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: from cryptography.utils import int_from_bytes Oct 20 08:52:06.496043 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 08:52:06.496043 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: from cryptography.utils import int_from_bytes Oct 20 08:52:08.972520 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 20 08:52:08.973182 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 20 08:52:08.973273 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 20 08:52:08.973273 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 20 08:52:08.973657 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 20 08:52:08.973657 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 20 08:52:08.973796 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 20 08:52:08.973871 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 20 08:52:08.974043 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 20 08:52:08.974237 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.974336 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 20 08:52:08.974588 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.974683 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.974913 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.975004 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.975231 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.975403 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.975560 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.975655 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.975871 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.975969 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.976200 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.976351 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.976514 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.976614 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.976836 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.976934 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.977095 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.977192 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.977409 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.977505 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.977718 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.977822 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.978040 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.978141 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.978352 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.978447 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.978683 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.978795 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.978991 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.979084 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.979292 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.979378 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.979601 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.979702 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.979905 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.979997 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.980219 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.980398 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.980803 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.980903 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.981032 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.981116 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.981335 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.981422 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.981646 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.981788 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.981981 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.982084 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.982299 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.982399 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.982637 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.982788 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.982969 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.983094 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.983264 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.983370 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.983572 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.983669 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.983884 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.983979 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.984263 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.984355 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.984575 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.984671 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.984886 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.984971 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.985202 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.985341 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.985585 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.985681 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.985893 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.985980 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.986212 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.986307 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.986561 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.986676 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.986915 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.987013 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.987233 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.987321 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.987549 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.987676 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.987868 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.987956 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.988187 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.988275 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.988494 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.988594 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.988803 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.988890 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.989111 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.989199 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.989422 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.989508 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.989724 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.989809 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.990029 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.990117 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.990280 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.990373 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.990610 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.990714 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.990921 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.991007 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.991240 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.991325 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.991561 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.991673 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.991810 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.991907 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.992062 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.992156 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.992368 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.992455 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.992688 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.992774 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.992998 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.993097 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.993304 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.993392 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.993613 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.993705 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.993917 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.994011 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.994223 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.994310 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.994551 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.994643 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.994860 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.994958 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.995164 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.995261 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.995470 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.995555 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.995782 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.995877 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.996097 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.996192 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.996400 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.996491 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.996717 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.996816 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.997046 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.997144 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.36 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.997349 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.36 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.997442 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.36 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.997657 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.997746 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.36 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.997981 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.998067 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.998306 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.998391 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.998635 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.998768 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.998953 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.999040 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.999270 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.999356 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.999579 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.999668 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.999887 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:08.999974 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.000193 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.000279 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.000505 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.000604 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.000826 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.000914 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.001127 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.001291 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.001375 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.001454 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.001666 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.001760 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.001971 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.002071 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.002844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.002980 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.003153 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.003243 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.003460 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.003561 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.003767 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.003854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.004071 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.004158 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.004374 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.004469 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.004676 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.004761 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.004981 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.005066 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.005287 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.005375 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.005592 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.005686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.005899 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.005986 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.006201 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.006290 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.006525 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.006616 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.006831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.006919 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.007134 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.007223 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.007442 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.007533 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.007757 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.007854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.008063 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.008160 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.008366 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.008467 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.008671 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.008759 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.008979 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.009077 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.009293 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.009381 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.009595 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.009681 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.009899 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.009992 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.010213 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.010301 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.010538 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.010628 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.010844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.010933 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.011151 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.011244 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.011455 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.011547 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.011760 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.011848 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.012068 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.012163 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.012366 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.012469 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.012671 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.012765 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.012978 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.013111 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.013281 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.013379 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.013584 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.013671 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.013883 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.013973 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.014184 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.014286 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.014505 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.014607 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.014813 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.014900 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.015116 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.015212 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.015417 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.015506 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.015716 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.015802 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.016016 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.016196 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.016335 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.016421 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.016641 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.016729 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.016945 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.017035 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.017247 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.017341 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.017571 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.017677 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.018483 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.018595 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:52:09.018836 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.018925 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.019174 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.019261 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.019488 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.019575 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.019821 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.019909 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.020154 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.020250 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.020468 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.020560 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.020781 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.020868 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.021093 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.021189 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.021409 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.021501 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.021718 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.021803 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.022029 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.022128 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.022337 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.022431 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.022675 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.022764 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.022993 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.023160 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.023317 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.023438 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.023631 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.023724 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.023946 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.024052 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.024261 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.024349 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.024567 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.024654 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.024880 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.024987 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.025203 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.025299 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.025519 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.025606 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.025831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.025922 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.026142 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.026230 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.026493 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.026584 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.026821 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.026927 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.027146 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.027243 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.027466 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.027568 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.027776 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.027866 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.028086 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.028182 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.028393 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.028481 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.028703 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.028793 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.029036 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.029131 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.029348 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.029451 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.029661 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.029761 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.029969 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.030057 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.030284 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.030380 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.030615 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.030706 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.030926 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.031016 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.031237 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.031336 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.031547 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.031634 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.031856 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.031951 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.032166 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.032262 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.032490 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.032588 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.032804 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.032891 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.033115 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.033226 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.033436 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.033555 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.033774 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.033862 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.034095 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.034185 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.034405 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.034539 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.034745 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.034843 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.035058 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.035148 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.035366 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.035454 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.035676 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.035776 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.035982 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.036069 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.036287 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.036380 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.036592 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.036679 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.036915 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.037068 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.037220 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.037306 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.037550 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.037676 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.037846 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.037936 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.038156 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.038243 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.038465 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.038590 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.038800 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.038895 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.039108 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.039201 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.039414 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.039513 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.039722 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.039809 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.040027 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.040116 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.040332 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.040427 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.040643 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.040729 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.040951 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.041045 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.041257 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.041344 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.041568 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.041653 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.041875 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.041961 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.042186 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.042274 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.042510 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.042600 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.042827 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.042923 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.043142 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.043251 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.043450 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.043576 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.043761 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.044049 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.044151 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.044367 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.044465 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.044682 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.044783 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.045014 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.045103 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.045343 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.045429 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.045647 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.045733 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.045971 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.046059 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.046292 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.046378 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.046632 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.046721 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.046939 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.047027 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.047246 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.047342 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.047550 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.047676 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.047854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.047942 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.048155 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.048241 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.048461 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.048557 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.048767 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.048860 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.049068 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.049638 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.049877 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.049979 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.050181 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.050268 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.050493 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.050580 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.050816 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.050915 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.051125 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.051211 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.051427 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.051525 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.051728 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.051823 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.052028 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.052114 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.052368 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.052456 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.052670 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.052759 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.053008 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.053095 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.053341 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.053428 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.053662 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.053760 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.053979 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.054066 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.054293 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.054381 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.054631 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.054721 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.054942 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.055028 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.055252 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.055338 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.055563 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.055658 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.055871 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.055957 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.056188 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.056287 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.056497 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.056584 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.056814 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.056901 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.057121 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.057216 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.057444 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.057541 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.057750 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.057848 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.058070 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.058171 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.058374 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.058485 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.058708 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.058794 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.059045 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.059144 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.059377 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.059466 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.059693 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.059780 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.060013 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.060109 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.060323 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.060409 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.060634 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.060721 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.060943 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.061030 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.061252 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.061338 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.061562 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.061649 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.061870 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.061964 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.062184 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.062270 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.062510 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.062613 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.062830 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.062919 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.063137 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.063223 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.063457 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.063543 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.063768 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.063867 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.064081 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.064167 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.064385 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.064471 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.064700 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.065354 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.065594 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.065682 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.065933 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.066036 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.066248 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.066343 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.066584 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.066680 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.066907 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.067011 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.067220 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.067314 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.067528 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.067618 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.067840 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.067937 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.068145 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.068238 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.068452 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.068547 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.068763 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.068862 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.069075 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.069164 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.069385 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.069473 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.069693 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.069780 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.069999 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.070086 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.070310 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.070403 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.070643 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.070734 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.070946 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.071044 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.071250 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.071337 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.071557 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.071645 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.071877 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.071963 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.072186 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.072273 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.072506 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.072594 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.072824 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.072910 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.073129 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.073224 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.073429 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.073517 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.073750 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.073847 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.074058 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.074165 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.074360 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.074455 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.074684 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.074770 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.074986 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.075086 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.075324 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.075411 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.075634 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.075721 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.075936 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.076032 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.076253 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.076339 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.076569 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.076663 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.076877 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.076964 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.077185 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.077270 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.077494 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.077587 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.077799 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.077887 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.078121 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.078211 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.078432 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.078577 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.078765 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.078855 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.079076 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.079163 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.079386 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.079472 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.079691 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.079779 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.080000 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.080088 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.080322 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.080419 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.080656 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.080745 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.080966 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.081052 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.081277 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.081373 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.081579 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.081673 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.081888 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.081982 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.082202 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.082406 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.082793 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.082884 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.083127 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.083214 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.083442 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.083537 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.083762 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.083849 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.084117 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.084227 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.084428 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.084526 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.084755 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.084852 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.085084 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.085171 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.085393 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.085479 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.085953 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.086062 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.086257 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.086344 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.086589 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.086681 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.086895 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.086999 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.087204 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.087291 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.087507 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.087639 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.087812 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.087907 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.088116 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.088203 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.088423 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.088510 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.088726 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.088822 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.089030 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.089120 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.089337 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.089465 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.089641 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.089735 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.089952 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.090039 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.090257 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.090344 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.090595 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.090684 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.090902 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.090990 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.091221 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.091317 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.091528 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.091618 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.091833 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.091919 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.092136 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.092233 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.092440 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.092527 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.092779 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.092879 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.093082 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.093183 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.093418 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.093550 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.093743 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.093830 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.094084 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.094194 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.094392 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.094525 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.094721 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.094821 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.095048 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.095136 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.095355 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.095442 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.095661 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.095747 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.095988 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.096083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.096303 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.096398 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.096611 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.096697 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.096913 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.097017 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.097223 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.097310 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.097530 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.097616 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.097840 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.097934 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.098146 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.098232 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.098451 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.098569 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.098782 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.098879 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.099086 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.099174 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.099402 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.099504 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.099709 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.099796 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.100013 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.100107 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.100313 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.100408 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.100621 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.100709 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.100927 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.101023 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.101230 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.101326 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.101536 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.101629 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.101838 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.101924 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.102150 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.102238 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.102455 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.102589 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.102779 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.102866 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.103082 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.103170 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.103388 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.103474 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.103695 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.103782 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.103995 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.104083 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.104317 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.104421 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.104627 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.104725 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.104930 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.105025 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.105229 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.105316 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.105536 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.105632 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.105837 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.105930 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.106141 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.106234 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.106445 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.106679 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.106775 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.106858 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.107089 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.107184 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.107417 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.107524 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.107697 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.107784 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.108002 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.108089 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.108306 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.108401 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.108607 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.108694 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.108914 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.109009 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.109216 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.109304 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.109521 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.109609 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.109820 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.109942 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.110122 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.110210 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.110421 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.110560 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.110766 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.110854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.111074 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.111163 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.111377 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.111465 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.112205 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.112312 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.112510 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.112598 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.112851 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.112941 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.113153 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.113250 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.113511 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.113605 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.113831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.113919 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.114130 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.114230 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.114447 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.114555 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.114784 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.114884 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.115093 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.115191 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.115405 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.115491 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.115708 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.115795 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.116019 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.116121 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.116330 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.116417 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.116641 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.116727 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.116946 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.117095 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.117247 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.117337 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.117567 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.117655 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.117877 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.117966 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.118187 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.118286 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.118529 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.118623 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.118857 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.118946 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.119179 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.119272 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.119486 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.119582 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.119794 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.119889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.120105 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.120193 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.120412 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.120498 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.120723 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.120810 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.121030 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.121116 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.121340 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.121435 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.121650 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.121737 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.121961 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.122047 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.122269 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.122363 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.122603 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.122703 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.122908 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.122997 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.123217 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.123353 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.123524 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.123611 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.123829 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.123915 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.124150 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.124238 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.124461 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.124559 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.124777 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.124908 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.125069 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.125157 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.125372 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.125458 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.125681 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.125769 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.125983 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.126070 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.126408 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.126517 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.126746 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.126848 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.127059 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.127155 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.127951 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.128049 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.128267 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.128364 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.128581 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.128668 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.128897 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.128984 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.129214 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.129310 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.129524 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.129611 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.129940 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.130039 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.130252 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.130348 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.130593 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.130687 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.130910 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.130998 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.131220 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.131329 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.131526 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.131614 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.131841 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.131928 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.132168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.132265 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.132475 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.132614 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.132780 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.132867 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.133085 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.133180 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.133408 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.133496 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.133711 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.133798 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.134034 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.134134 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.134458 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.134595 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.134782 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.134870 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.135089 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.135180 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.135500 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.135587 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.135814 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.135909 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.136124 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.136236 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.136448 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.136538 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.136769 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.136856 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.137094 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.137182 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.137399 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.137495 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.137721 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.137841 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.138024 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.138132 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.138347 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.138438 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.138699 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.138801 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.139010 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.139111 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.139327 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.139417 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.139632 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.139733 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.139956 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.140061 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.140276 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.140365 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.140591 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.140679 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.140913 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.141010 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.141323 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.141418 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.141645 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.141734 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.141957 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.142046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.142274 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.142361 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.142622 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.142714 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.143000 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.143154 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.143290 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.143545 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.143694 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.143927 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.144074 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.144221 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.144489 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.144624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.144841 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.144988 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.145157 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.145271 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.145471 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.145587 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.145782 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.145906 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.146089 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.146223 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.146394 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.146562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.146743 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.146876 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.147048 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.147186 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.147371 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.147515 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.147688 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.147823 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.148012 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.148145 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.148333 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.148465 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.148641 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.148778 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.148965 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.149105 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.149274 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.149415 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.149585 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.149742 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.149890 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.150056 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.150196 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.150366 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.150560 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.150700 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.150868 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.151011 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.151188 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.151357 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.151517 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.151665 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.151846 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.151988 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.152157 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.152321 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.152489 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.152654 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.152812 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.152967 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.153134 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.153304 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.153444 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.153635 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.153759 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.153958 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.154080 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.154273 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.154384 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.154619 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.154710 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.154945 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.155044 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.155252 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.155343 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.155560 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.155649 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.155873 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.155970 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.156177 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.156264 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.156481 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.156569 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.156783 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.156880 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.157084 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.157173 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.157386 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.157473 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.157693 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.157790 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.158004 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.158099 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.158308 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.158401 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.158671 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.158771 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.158979 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.159069 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.159284 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.159373 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.159588 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.159676 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.159895 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.159981 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.160202 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.160290 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.160527 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.160615 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.160839 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.160925 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.161153 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.161247 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.161467 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.161557 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.161782 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.161870 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.162096 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.162185 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.162411 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.162538 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.162749 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.162837 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.163063 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.163163 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.163370 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.163466 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.163685 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.163773 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.163999 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.164095 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.164310 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.164399 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.164629 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.164826 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.164958 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.165157 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.165246 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.165473 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.165763 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.165853 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.166089 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.166217 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.166404 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.166529 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.166741 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.166830 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.167052 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.167145 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.167365 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.167454 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.167679 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.167776 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.167987 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.168076 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.168299 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.168395 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.168615 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.168712 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.168932 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.169100 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.169262 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.169395 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.169583 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.169731 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.169893 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.170030 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.170206 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.170341 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.170540 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.170646 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.170868 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.170956 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.171185 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.171275 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.171490 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.171578 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.171805 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.171894 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.172119 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.172207 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.172432 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.172533 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.172743 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.172832 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.173052 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.173153 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.173363 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.173465 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.173673 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.173763 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.173983 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.174615 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.174837 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.174926 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.175155 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.175252 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.175465 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.175573 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.175773 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.175862 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.176090 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.176180 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.176405 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.176503 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.176714 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.176809 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.177029 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.177118 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.177345 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.177435 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.177660 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.177862 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.177977 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.178175 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.178301 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.178528 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.178848 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.178945 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.179168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.179265 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.179480 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.179570 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.179804 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.179900 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.180130 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.180227 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.180436 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.180526 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.180742 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.180830 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.181054 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.181150 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.181359 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.181446 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.181661 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.181748 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.181971 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.182066 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.182278 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.182374 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.182603 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.182694 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.182924 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.183027 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.183228 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.183314 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.183534 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.183759 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.183858 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.184106 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.184194 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.184421 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.184666 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.184755 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.185013 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.185156 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.185329 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.185419 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.185650 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.185753 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.185970 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.186059 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.186275 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.186362 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.186603 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.186695 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.186922 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.187011 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.187228 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.187315 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.187531 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.187632 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.187837 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.187924 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.188145 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.188231 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.188450 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.188552 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.188753 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.188848 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.189060 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.189148 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.189365 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.189470 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.189668 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.189815 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.190558 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.190653 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.190869 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.190968 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.191197 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.191286 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.191506 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.191600 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.191811 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.191899 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.192115 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.192201 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.192424 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.192512 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.192728 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.192818 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.193033 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.193123 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.193335 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.193423 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.193639 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.193736 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.193943 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.194031 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.194245 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.194332 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.194580 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.194673 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.194888 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.194980 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.195201 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.195299 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.195521 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.195646 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.195827 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.195914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.196131 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.196228 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.196448 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.196536 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.196752 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.196839 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.197058 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.197155 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.197370 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.197457 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.197677 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.197763 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.198001 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.198099 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.198322 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.198409 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.198663 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.198873 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.199020 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.199210 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.199345 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.199532 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.199838 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.199934 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.200179 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.200268 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.200488 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.200576 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.200796 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.200884 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.201104 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.201193 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.201427 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.201515 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.201739 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.201828 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.202048 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.202136 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.202353 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.202455 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.202689 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.202791 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.203000 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.203088 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.203311 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.203399 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.203616 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.203755 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.203925 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.204013 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.204232 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.204318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.204541 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.204638 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.204849 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.204935 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.205155 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.205249 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.205489 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.205578 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.205804 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.205901 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.206113 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.206201 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.206424 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.206547 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.206761 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.206850 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.207078 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.207168 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.207393 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.207518 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.207713 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.207802 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.208025 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.208121 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.208339 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.208427 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.208652 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.208747 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.208963 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.209058 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.209277 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.209363 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.209590 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.209686 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.209904 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.210069 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.210210 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.210297 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.210562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.211144 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.211248 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.211457 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.211546 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.211778 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.212103 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.212190 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.212419 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.212516 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.212747 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.212836 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.213072 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.213166 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.213384 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.213472 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.213708 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.213840 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.214021 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.214127 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.214339 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.214427 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.214691 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.214782 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.215008 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.215109 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.215342 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.215431 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.215648 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.215735 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.215958 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.216058 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.216267 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.216356 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.216578 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.216672 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.216898 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.216998 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.217205 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.217293 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.217509 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.217596 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.217816 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.217943 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.218127 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.218228 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.218434 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.218552 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.218764 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.218854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.219072 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.219160 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.219377 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.219464 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.219687 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.219783 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.219991 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.220079 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.220295 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.220398 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.220601 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.220688 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.220904 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.221010 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.221234 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.221330 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.221542 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.221629 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.221846 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.221932 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.222148 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.222308 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.222451 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.222603 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.222785 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.222875 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.223090 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.223186 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.223394 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.223482 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.223700 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.223789 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.224008 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.224101 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.224317 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.224406 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.224649 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.224736 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.224969 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.225059 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.225283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.225370 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.225591 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.225678 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.225908 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.225996 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.226227 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.226314 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.226575 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.226672 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.226888 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.226979 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.227221 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.227310 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.227533 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.227630 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.227861 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.227955 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.228158 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.228252 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.228470 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.228569 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.228778 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.228868 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.229090 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.229178 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.229399 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.229495 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.229713 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.229810 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.230025 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.230121 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.230349 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.230445 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.230702 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.230849 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.231015 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.231237 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.231328 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.231598 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.231676 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.231950 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.232027 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.232263 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.232354 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.232587 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.232695 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.232911 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.232994 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.233225 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.233312 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.233540 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.233640 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.233851 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.233997 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.234163 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.234334 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.234522 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.234702 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.234812 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.235032 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.235140 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.235365 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.235454 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.235694 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.235783 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.236033 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.236142 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.236348 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.236437 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.237188 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.237286 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.237502 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.237592 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.237831 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.237923 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.238161 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.238249 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.238540 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.238748 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.238864 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.239123 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.239212 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.239469 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.239571 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.239828 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.239926 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.240146 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.240242 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.240456 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.240659 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:52:09.240747 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103837) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 20 08:52:09.240924 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103836) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 20 08:52:09.393179 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO dbcounter [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Registered counter for database nova_api Oct 20 08:52:09.393683 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO dbcounter [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Registered counter for database nova_api Oct 20 08:52:09.415311 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG oslo_db.sqlalchemy.engines [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103836) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:52:09.417027 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_db.sqlalchemy.engines [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103837) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:52:09.421989 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writer thread running {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:52:09.423800 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writer thread running {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:52:09.479316 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.context [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dd273305-b4a6-40de-9d39-cde54b55a76f(cell1) {{(pid=103836) load_cells /opt/stack/nova/nova/context.py:464}} Oct 20 08:52:09.480947 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.context [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dd273305-b4a6-40de-9d39-cde54b55a76f(cell1) {{(pid=103837) load_cells /opt/stack/nova/nova/context.py:464}} Oct 20 08:52:09.483119 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG oslo_concurrency.lockutils [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103836) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:52:09.483599 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG oslo_concurrency.lockutils [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103836) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:52:09.484774 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:52:09.485148 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG oslo_concurrency.lockutils [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103836) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:52:09.485252 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:52:09.485894 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG oslo_concurrency.lockutils [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103836) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:52:09.486344 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG oslo_concurrency.lockutils [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103836) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:52:09.487452 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:52:09.488194 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:52:09.488641 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:52:09.488872 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG oslo_concurrency.lockutils [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103836) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:52:09.490621 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:52:09.490881 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO dbcounter [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Registered counter for database nova_cell0 Oct 20 08:52:09.492588 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO dbcounter [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Registered counter for database nova_cell0 Oct 20 08:52:09.494918 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO dbcounter [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Registered counter for database nova_cell1 Oct 20 08:52:09.496707 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO dbcounter [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Registered counter for database nova_cell1 Oct 20 08:52:09.502524 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG oslo_db.sqlalchemy.engines [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103836) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:52:09.503835 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG oslo_db.sqlalchemy.engines [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103836) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:52:09.506588 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_db.sqlalchemy.engines [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103837) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:52:09.508262 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_db.sqlalchemy.engines [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103837) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:52:09.510513 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writer thread running {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:52:09.514285 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writer thread running {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:52:09.515437 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writer thread running {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:52:09.516165 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writer thread running {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:52:09.660399 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO dbcounter [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Registered counter for database nova_cell0 Oct 20 08:52:09.663707 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO dbcounter [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Registered counter for database nova_cell0 Oct 20 08:52:09.669238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG oslo_db.sqlalchemy.engines [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103836) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:52:09.671134 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_db.sqlalchemy.engines [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103837) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:52:09.674796 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writer thread running {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:52:09.676313 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writer thread running {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:52:10.005149 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: WARNING keystonemiddleware.auth_token [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 20 08:52:10.007825 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: WARNING keystonemiddleware.auth_token [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 20 08:52:10.012292 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: WARNING keystonemiddleware.auth_token [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 20 08:52:10.014993 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: WARNING keystonemiddleware.auth_token [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 20 08:52:10.350448 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: WARNING keystonemiddleware.auth_token [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 20 08:52:10.355416 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: WARNING keystonemiddleware.auth_token [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 20 08:52:10.356296 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: WARNING keystonemiddleware.auth_token [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 20 08:52:10.361329 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: WARNING keystonemiddleware.auth_token [None req-c375bc1b-976d-4a3e-ae99-c87e5ee3dffb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 20 08:52:10.361580 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fde78ed9c0 pid: 103837 (default app) Oct 20 08:52:10.363628 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:52:10.364829 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] 10.222.0.36 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001724 Oct 20 08:52:10.365546 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 788 bytes} [Thu Oct 20 08:52:10 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:52:10.366644 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fde78ed9c0 pid: 103836 (default app) Oct 20 08:53:31.010633 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:31.013642 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003510 Oct 20 08:53:31.014041 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 2/2] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:53:31 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:53:31.026625 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writing DB stats nova_api:SELECT=3 {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:53:31.027721 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writing DB stats nova_cell0:SELECT=3 {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:53:31.028844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writing DB stats nova_cell1:SELECT=3 {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:53:31.029642 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:53:31.877638 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-545fe775-7184-46a2-925e-394a10b73543 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:31.910032 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-545fe775-7184-46a2-925e-394a10b73543 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.888520 Oct 20 08:53:31.910797 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 1/3] 10.222.0.36 () {60 vars in 1292 bytes} [Thu Oct 20 08:53:31 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 890 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:53:34.517157 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:34.518156 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-d5182f42-e89e-4281-b118-8fb3f42cbbf8 None None] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001415 Oct 20 08:53:34.518743 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 3/4] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:53:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:53:34.962271 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-2507a27b-6928-4e1b-975b-f80db4cd66bb admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "cirros256", "vcpus": 1, "swap": 0, "os-flavor-access:is_public": true, "ram": 256, "id": "c1", "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 1, "rxtx_factor": 1.0}} {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:35.010035 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-2507a27b-6928-4e1b-975b-f80db4cd66bb admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.485490 Oct 20 08:53:35.010612 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 2/5] 10.222.0.36 () {64 vars in 1283 bytes} [Thu Oct 20 08:53:34 2022] POST /compute/v2.1/flavors => generated 407 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:53:35.024328 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writing DB stats nova_api:SELECT=3 {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:53:35.026363 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writing DB stats nova_cell1:SELECT=3 {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:53:35.027947 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writing DB stats nova_cell0:SELECT=3 {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:53:35.029042 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:53:35.235407 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-89268c32-a704-47b5-915c-911792ec75a6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:35.310099 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-89268c32-a704-47b5-915c-911792ec75a6 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.291018 Oct 20 08:53:35.310718 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 4/6] 10.222.0.36 () {64 vars in 1336 bytes} [Thu Oct 20 08:53:35 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:53:37.922107 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-2507a27b-6928-4e1b-975b-f80db4cd66bb admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:37.923832 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-2507a27b-6928-4e1b-975b-f80db4cd66bb admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002175 Oct 20 08:53:37.929671 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 3/7] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:53:37 2022] GET /compute/v2.1 => generated 388 bytes in 8 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:53:38.588451 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-8bf83cb4-f5f3-41eb-916e-5f908c37a43a admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d1", "ram": 512, "name": "ds512M", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "os-flavor-access:is_public": true, "vcpus": 1, "disk": 5}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:38.630817 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-8bf83cb4-f5f3-41eb-916e-5f908c37a43a admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.699430 Oct 20 08:53:38.634550 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 5/8] 10.222.0.36 () {64 vars in 1283 bytes} [Thu Oct 20 08:53:37 2022] POST /compute/v2.1/flavors => generated 404 bytes in 701 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:53:38.643903 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-7e47b365-de2b-4574-8818-deea5af08f9e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:38.684124 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-7e47b365-de2b-4574-8818-deea5af08f9e admin admin] 10.222.0.36 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.044374 Oct 20 08:53:38.685505 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 4/9] 10.222.0.36 () {64 vars in 1336 bytes} [Thu Oct 20 08:53:38 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:53:41.260373 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-8bf83cb4-f5f3-41eb-916e-5f908c37a43a admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:41.261405 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-8bf83cb4-f5f3-41eb-916e-5f908c37a43a admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001443 Oct 20 08:53:41.261774 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 6/10] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:53:41 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:53:41.677587 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-6a593550-1f2d-460e-a831-359025df3191 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 10, "os-flavor-access:is_public": true, "vcpus": 1, "id": "d2", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 1024, "name": "ds1G"}} {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:41.700426 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-6a593550-1f2d-460e-a831-359025df3191 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.431704 Oct 20 08:53:41.700984 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 5/11] 10.222.0.36 () {64 vars in 1283 bytes} [Thu Oct 20 08:53:41 2022] POST /compute/v2.1/flavors => generated 404 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:53:41.713738 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-d64f5583-e71a-4b7f-a2c9-7a137df8a2c7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:41.741730 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-d64f5583-e71a-4b7f-a2c9-7a137df8a2c7 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031842 Oct 20 08:53:41.742284 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 7/12] 10.222.0.36 () {64 vars in 1336 bytes} [Thu Oct 20 08:53:41 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:53:44.353007 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-6a593550-1f2d-460e-a831-359025df3191 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:44.353931 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-6a593550-1f2d-460e-a831-359025df3191 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001451 Oct 20 08:53:44.354336 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 6/13] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:53:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:53:44.760952 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-734f2082-b4ff-4a86-b3ca-1a61482ffd55 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds2G", "disk": 10, "swap": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "vcpus": 2, "id": "d3", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:44.781618 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-734f2082-b4ff-4a86-b3ca-1a61482ffd55 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.421279 Oct 20 08:53:44.782401 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 8/14] 10.222.0.36 () {64 vars in 1283 bytes} [Thu Oct 20 08:53:44 2022] POST /compute/v2.1/flavors => generated 404 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:53:44.793909 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-520b1c28-affa-411c-94d3-551b80819ff8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:44.821409 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-520b1c28-affa-411c-94d3-551b80819ff8 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030982 Oct 20 08:53:44.822160 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 7/15] 10.222.0.36 () {64 vars in 1336 bytes} [Thu Oct 20 08:53:44 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 08:53:47.417642 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-734f2082-b4ff-4a86-b3ca-1a61482ffd55 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:47.419219 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-734f2082-b4ff-4a86-b3ca-1a61482ffd55 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002194 Oct 20 08:53:47.419778 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 9/16] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:53:47 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:53:47.922645 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-8520aea9-0dc5-4585-bed0-d916fc3d3567 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 4, "rxtx_factor": 1.0, "disk": 20, "ram": 4096, "name": "ds4G", "swap": 0, "id": "d4"}} {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:47.942510 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-8520aea9-0dc5-4585-bed0-d916fc3d3567 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.516518 Oct 20 08:53:47.943078 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 8/17] 10.222.0.36 () {64 vars in 1283 bytes} [Thu Oct 20 08:53:47 2022] POST /compute/v2.1/flavors => generated 404 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 20 08:53:47.960079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-8638afbb-af7c-461d-87a9-9ea254d144f1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:48.005585 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-8638afbb-af7c-461d-87a9-9ea254d144f1 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.053236 Oct 20 08:53:48.006875 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 10/18] 10.222.0.36 () {64 vars in 1336 bytes} [Thu Oct 20 08:53:47 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:53:50.675138 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-8520aea9-0dc5-4585-bed0-d916fc3d3567 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:50.676193 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-8520aea9-0dc5-4585-bed0-d916fc3d3567 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001543 Oct 20 08:53:50.676615 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 9/19] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:53:50 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:53:51.112650 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-a6c21f55-0a2f-4867-a28b-24e9df982332 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:51.138341 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-a6c21f55-0a2f-4867-a28b-24e9df982332 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.455546 Oct 20 08:53:51.139007 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 11/20] 10.222.0.36 () {60 vars in 1292 bytes} [Thu Oct 20 08:53:50 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 08:53:53.729564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-8520aea9-0dc5-4585-bed0-d916fc3d3567 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:53.730630 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-8520aea9-0dc5-4585-bed0-d916fc3d3567 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001487 Oct 20 08:53:53.731055 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 10/21] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:53:53 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:53:54.125141 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-8944a6f7-6c76-4f10-8c77-a0cc167a31fd admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.tiny", "os-flavor-access:is_public": true, "vcpus": 1, "ram": 512, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "rxtx_factor": 1.0, "id": "1", "disk": 1}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:54.147549 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-8944a6f7-6c76-4f10-8c77-a0cc167a31fd admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.408918 Oct 20 08:53:54.148096 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 12/22] 10.222.0.36 () {64 vars in 1283 bytes} [Thu Oct 20 08:53:53 2022] POST /compute/v2.1/flavors => generated 402 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:53:54.160534 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-964c512e-d840-4a1e-b2bf-7335678006dc admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:54.186495 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-964c512e-d840-4a1e-b2bf-7335678006dc admin admin] 10.222.0.36 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029526 Oct 20 08:53:54.187079 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 11/23] 10.222.0.36 () {64 vars in 1333 bytes} [Thu Oct 20 08:53:54 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:53:56.773372 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-8944a6f7-6c76-4f10-8c77-a0cc167a31fd admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:56.774374 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-8944a6f7-6c76-4f10-8c77-a0cc167a31fd admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001418 Oct 20 08:53:56.774814 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 13/24] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:53:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:53:57.148582 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-ba5d5eaa-64b9-4011-b4fb-34baac6135de admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "2", "ram": 2048, "name": "m1.small", "rxtx_factor": 1.0, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 20, "swap": 0, "os-flavor-access:is_public": true}} {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:57.168014 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-ba5d5eaa-64b9-4011-b4fb-34baac6135de admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.386626 Oct 20 08:53:57.168647 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 12/25] 10.222.0.36 () {64 vars in 1283 bytes} [Thu Oct 20 08:53:56 2022] POST /compute/v2.1/flavors => generated 405 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:53:57.180612 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-1312aff0-390e-4fc3-a6a1-02980dff41db admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:53:57.203733 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-1312aff0-390e-4fc3-a6a1-02980dff41db admin admin] 10.222.0.36 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026750 Oct 20 08:53:57.204474 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 14/26] 10.222.0.36 () {64 vars in 1333 bytes} [Thu Oct 20 08:53:57 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:53:59.822568 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-ba5d5eaa-64b9-4011-b4fb-34baac6135de admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:53:59.823596 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-ba5d5eaa-64b9-4011-b4fb-34baac6135de admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001492 Oct 20 08:53:59.824212 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 13/27] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:53:59 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:00.325207 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-f4d2b623-bbea-43c2-af35-d9d0d0034f16 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.medium", "id": "3", "swap": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "vcpus": 2, "disk": 40, "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:54:00.347717 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-f4d2b623-bbea-43c2-af35-d9d0d0034f16 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.517100 Oct 20 08:54:00.348269 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 15/28] 10.222.0.36 () {64 vars in 1283 bytes} [Thu Oct 20 08:53:59 2022] POST /compute/v2.1/flavors => generated 406 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:54:00.361429 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-dfa14f30-7f6e-4883-a878-c3e2316beb58 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:54:00.397722 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-dfa14f30-7f6e-4883-a878-c3e2316beb58 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.040167 Oct 20 08:54:00.398306 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 14/29] 10.222.0.36 () {64 vars in 1333 bytes} [Thu Oct 20 08:54:00 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:54:03.009148 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-f4d2b623-bbea-43c2-af35-d9d0d0034f16 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:03.010082 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-f4d2b623-bbea-43c2-af35-d9d0d0034f16 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001465 Oct 20 08:54:03.010504 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 16/30] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:54:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:03.444795 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-4d12af07-2d1d-405b-8140-7f1044d0da0a admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "4", "vcpus": 4, "ram": 8192, "disk": 80, "name": "m1.large", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "swap": 0}} {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:54:03.450287 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:54:03.465253 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-4d12af07-2d1d-405b-8140-7f1044d0da0a admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.448122 Oct 20 08:54:03.465831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 15/31] 10.222.0.36 () {64 vars in 1283 bytes} [Thu Oct 20 08:54:03 2022] POST /compute/v2.1/flavors => generated 405 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 20 08:54:03.478149 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-49338131-4eeb-4e43-8d12-ac6251914073 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:54:03.503062 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-49338131-4eeb-4e43-8d12-ac6251914073 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028470 Oct 20 08:54:03.510144 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 17/32] 10.222.0.36 () {64 vars in 1333 bytes} [Thu Oct 20 08:54:03 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:54:06.123584 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-4d12af07-2d1d-405b-8140-7f1044d0da0a admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:06.125251 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-4d12af07-2d1d-405b-8140-7f1044d0da0a admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002103 Oct 20 08:54:06.125809 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 16/33] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:54:06 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:06.547208 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-88fe609e-6e4e-4808-a888-7c0c88a9ab87 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 16384, "name": "m1.xlarge", "swap": 0, "disk": 160, "rxtx_factor": 1.0, "vcpus": 8, "os-flavor-access:is_public": true, "id": "5", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:54:06.557057 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:54:06.575257 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-88fe609e-6e4e-4808-a888-7c0c88a9ab87 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.443264 Oct 20 08:54:06.578609 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 18/34] 10.222.0.36 () {64 vars in 1283 bytes} [Thu Oct 20 08:54:06 2022] POST /compute/v2.1/flavors => generated 408 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:54:06.588562 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-a42fdcdf-3af2-4767-8e46-2c6b14967249 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:54:06.615997 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-a42fdcdf-3af2-4767-8e46-2c6b14967249 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031757 Oct 20 08:54:06.616761 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 17/35] 10.222.0.36 () {64 vars in 1333 bytes} [Thu Oct 20 08:54:06 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 08:54:23.113725 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-88fe609e-6e4e-4808-a888-7c0c88a9ab87 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:23.114845 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-88fe609e-6e4e-4808-a888-7c0c88a9ab87 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001486 Oct 20 08:54:23.115242 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 19/36] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:54:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:23.125693 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:54:23.501489 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-32ac0b41-9319-434d-a729-b64ed26735bd admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:23.519611 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-32ac0b41-9319-434d-a729-b64ed26735bd admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.398354 Oct 20 08:54:23.520143 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 18/37] 10.222.0.36 () {60 vars in 1292 bytes} [Thu Oct 20 08:54:23 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 08:54:24.813860 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-88fe609e-6e4e-4808-a888-7c0c88a9ab87 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:24.814894 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-88fe609e-6e4e-4808-a888-7c0c88a9ab87 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001408 Oct 20 08:54:24.815290 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 20/38] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:54:24 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:25.326833 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-70163bac-6194-4a0f-9d4c-72806fb570b6 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:25.332049 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.wsgi [None req-70163bac-6194-4a0f-9d4c-72806fb570b6 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-70163bac-6194-4a0f-9d4c-72806fb570b6 admin admin] Request method failure captured: Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Accept: application/json Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Accept-Encoding: gzip, deflate Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Connection: keep-alive Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Content-Length: 0 Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Host: 10.222.0.36 Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Openstack-Api-Version: compute 2.92 Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Openstack-System-Scope: None Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: User-Agent: python-novaclient Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Auth-Token: gAAAAABjUQzAhA9pplVJaC5qaO3Q1lLNki_p6yfw9Ci69HExqjTaGsm9NknDFZFbu6S7X-U3Ivg6locvtG4Risv4ctkOUXYGvbfkMpXdEKwJybZbx7SpmuHTdbuktOsktbua7k5vbl6Nb__7gJZCdDOV_Ez3QJmqLaC0HXYFic359OQAFljJaDA Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Domain-Id: None Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Domain-Name: None Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Identity-Status: Confirmed Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Is-Admin-Project: True Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Openstack-Nova-Api-Version: 2.92 Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Domain-Id: default Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Domain-Name: Default Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Name: admin Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Role: admin,reader,member Oct 20 08:54:25.335512 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Roles: admin,reader,member Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_602743e1ebcd495bbe3a73b9f8abba4e", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}] Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Tenant: admin Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Tenant-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Tenant-Name: admin Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User: admin Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Domain-Id: default Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Domain-Name: Default Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Id: 9804008bc9774f7b96bc93c72ddebd2f Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Name: admin Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: method: > Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: exception: Flavor 42 could not be found. Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: action_args: {'id': '42'} Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 08:54:25.337159 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:25.338238 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:25.339817 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-70163bac-6194-4a0f-9d4c-72806fb570b6 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103836) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 08:54:25.341729 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-70163bac-6194-4a0f-9d4c-72806fb570b6 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.387378 Oct 20 08:54:25.341729 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 19/39] 10.222.0.36 () {60 vars in 1190 bytes} [Thu Oct 20 08:54:24 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 388 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 20 08:54:25.347236 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-e89b36f1-0e20-4784-8722-3c2d1b5057e5 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:25.370308 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-e89b36f1-0e20-4784-8722-3c2d1b5057e5 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026588 Oct 20 08:54:25.370879 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 21/40] 10.222.0.36 () {60 vars in 1210 bytes} [Thu Oct 20 08:54:25 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 08:54:25.381475 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-8e7d14ab-90a0-422e-8914-693323f4fcd6 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:25.403568 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-8e7d14ab-90a0-422e-8914-693323f4fcd6 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026117 Oct 20 08:54:25.404112 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 20/41] 10.222.0.36 () {60 vars in 1210 bytes} [Thu Oct 20 08:54:25 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 08:54:25.413524 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-78518997-3868-4f89-bc36-4a070f3040b4 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:25.419360 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.wsgi [None req-78518997-3868-4f89-bc36-4a070f3040b4 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-78518997-3868-4f89-bc36-4a070f3040b4 admin admin] Request method failure captured: Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Accept: application/json Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Accept-Encoding: gzip, deflate Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Connection: keep-alive Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Content-Length: 0 Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Host: 10.222.0.36 Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Openstack-Api-Version: compute 2.92 Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Openstack-System-Scope: None Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: User-Agent: python-novaclient Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Auth-Token: gAAAAABjUQzAhA9pplVJaC5qaO3Q1lLNki_p6yfw9Ci69HExqjTaGsm9NknDFZFbu6S7X-U3Ivg6locvtG4Risv4ctkOUXYGvbfkMpXdEKwJybZbx7SpmuHTdbuktOsktbua7k5vbl6Nb__7gJZCdDOV_Ez3QJmqLaC0HXYFic359OQAFljJaDA Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Domain-Id: None Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Domain-Name: None Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Identity-Status: Confirmed Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Is-Admin-Project: True Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Openstack-Nova-Api-Version: 2.92 Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Domain-Id: default Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Domain-Name: Default Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Name: admin Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Role: admin,reader,member Oct 20 08:54:25.422562 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Roles: admin,reader,member Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_602743e1ebcd495bbe3a73b9f8abba4e", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}] Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Tenant: admin Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Tenant-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Tenant-Name: admin Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User: admin Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Domain-Id: default Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Domain-Name: Default Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Id: 9804008bc9774f7b96bc93c72ddebd2f Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Name: admin Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: method: > Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: exception: Flavor 42 could not be found. Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: action_args: {'id': '42'} Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 08:54:25.423914 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:25.424889 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:25.426022 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-78518997-3868-4f89-bc36-4a070f3040b4 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103837) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 08:54:25.427060 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-78518997-3868-4f89-bc36-4a070f3040b4 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.013917 Oct 20 08:54:25.427060 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 22/42] 10.222.0.36 () {60 vars in 1190 bytes} [Thu Oct 20 08:54:25 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 20 08:54:26.631303 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-8e7d14ab-90a0-422e-8914-693323f4fcd6 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:26.632269 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-8e7d14ab-90a0-422e-8914-693323f4fcd6 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001492 Oct 20 08:54:26.632676 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 21/43] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:54:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:27.151640 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-9ef84ae2-8f7e-4a09-8347-cbe6ae57a875 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:27.157251 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.wsgi [None req-9ef84ae2-8f7e-4a09-8347-cbe6ae57a875 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-9ef84ae2-8f7e-4a09-8347-cbe6ae57a875 admin admin] Request method failure captured: Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Accept: application/json Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Accept-Encoding: gzip, deflate Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Connection: keep-alive Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Content-Length: 0 Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Host: 10.222.0.36 Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Openstack-Api-Version: compute 2.92 Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Openstack-System-Scope: None Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: User-Agent: python-novaclient Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Auth-Token: gAAAAABjUQzCz4B61e6PX6DiW8gox5xHtlaMeY7ODIpIguHzCy8st0x2tScNMVSa_u-47WuRn4hOpFI2Yl9nGyc91QxSDeD7TWOcGVcb60pZ8vXPDsUWxhjYT-C6wCBgI-FAJGozSh3eo9AYyIqaJ0GVSlsLNtBnrLx-LamPxHshGp_UhGHybs0 Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Domain-Id: None Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Domain-Name: None Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Identity-Status: Confirmed Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Is-Admin-Project: True Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Openstack-Nova-Api-Version: 2.92 Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Domain-Id: default Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Domain-Name: Default Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Name: admin Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Role: admin,reader,member Oct 20 08:54:27.158808 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Roles: admin,reader,member Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_602743e1ebcd495bbe3a73b9f8abba4e", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}] Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Tenant: admin Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Tenant-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Tenant-Name: admin Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User: admin Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Domain-Id: default Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Domain-Name: Default Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Id: 9804008bc9774f7b96bc93c72ddebd2f Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Name: admin Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: method: > Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: exception: Flavor 84 could not be found. Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: action_args: {'id': '84'} Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 08:54:27.160283 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:27.161079 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:27.162168 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-9ef84ae2-8f7e-4a09-8347-cbe6ae57a875 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103837) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 08:54:27.163219 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-9ef84ae2-8f7e-4a09-8347-cbe6ae57a875 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.388937 Oct 20 08:54:27.163219 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 23/44] 10.222.0.36 () {60 vars in 1190 bytes} [Thu Oct 20 08:54:26 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 390 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 20 08:54:27.169850 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-de477a4e-3aaf-4fdc-83cf-267d5a171e72 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:27.184518 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-de477a4e-3aaf-4fdc-83cf-267d5a171e72 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018147 Oct 20 08:54:27.184999 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 22/45] 10.222.0.36 () {60 vars in 1210 bytes} [Thu Oct 20 08:54:27 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 08:54:27.194153 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-99fead00-80f8-46f7-b6e1-588dedcc5dd0 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:27.209528 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-99fead00-80f8-46f7-b6e1-588dedcc5dd0 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018874 Oct 20 08:54:27.210039 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 24/46] 10.222.0.36 () {60 vars in 1210 bytes} [Thu Oct 20 08:54:27 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 08:54:27.219699 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-5a94e7d7-6585-4d64-9209-56a5fd9b49f1 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:27.224564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.wsgi [None req-5a94e7d7-6585-4d64-9209-56a5fd9b49f1 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-5a94e7d7-6585-4d64-9209-56a5fd9b49f1 admin admin] Request method failure captured: Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Accept: application/json Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Accept-Encoding: gzip, deflate Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Connection: keep-alive Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Content-Length: 0 Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Host: 10.222.0.36 Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Openstack-Api-Version: compute 2.92 Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Openstack-System-Scope: None Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: User-Agent: python-novaclient Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Auth-Token: gAAAAABjUQzCz4B61e6PX6DiW8gox5xHtlaMeY7ODIpIguHzCy8st0x2tScNMVSa_u-47WuRn4hOpFI2Yl9nGyc91QxSDeD7TWOcGVcb60pZ8vXPDsUWxhjYT-C6wCBgI-FAJGozSh3eo9AYyIqaJ0GVSlsLNtBnrLx-LamPxHshGp_UhGHybs0 Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Domain-Id: None Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Domain-Name: None Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Identity-Status: Confirmed Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Is-Admin-Project: True Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Openstack-Nova-Api-Version: 2.92 Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Domain-Id: default Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Domain-Name: Default Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Name: admin Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Role: admin,reader,member Oct 20 08:54:27.226046 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Roles: admin,reader,member Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_602743e1ebcd495bbe3a73b9f8abba4e", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}] Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Tenant: admin Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Tenant-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Tenant-Name: admin Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User: admin Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Domain-Id: default Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Domain-Name: Default Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Id: 9804008bc9774f7b96bc93c72ddebd2f Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Name: admin Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: method: > Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: exception: Flavor 84 could not be found. Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: action_args: {'id': '84'} Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 08:54:27.227624 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:27.228564 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:27.230083 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-5a94e7d7-6585-4d64-9209-56a5fd9b49f1 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103836) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 08:54:27.231337 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-5a94e7d7-6585-4d64-9209-56a5fd9b49f1 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011719 Oct 20 08:54:27.231337 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 23/47] 10.222.0.36 () {60 vars in 1190 bytes} [Thu Oct 20 08:54:27 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 20 08:54:28.436652 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-99fead00-80f8-46f7-b6e1-588dedcc5dd0 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:28.437652 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-99fead00-80f8-46f7-b6e1-588dedcc5dd0 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001420 Oct 20 08:54:28.438030 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 25/48] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:54:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:28.956373 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-84b042b2-eec1-4dbb-9e9e-f76bef42c1d3 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:28.962220 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.wsgi [None req-84b042b2-eec1-4dbb-9e9e-f76bef42c1d3 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-84b042b2-eec1-4dbb-9e9e-f76bef42c1d3 admin admin] Request method failure captured: Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Accept: application/json Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Accept-Encoding: gzip, deflate Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Connection: keep-alive Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Content-Length: 0 Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Host: 10.222.0.36 Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Openstack-Api-Version: compute 2.92 Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Openstack-System-Scope: None Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: User-Agent: python-novaclient Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Auth-Token: gAAAAABjUQzEozokudtQdyINDxozNjJHlkE4oyrepjyZH05aLeLUq5ro7WeC8-7_qjkvDzc48fvc0t6fx3H93P70oSy5bdUfbqKQP8-v8-GnuExSfzlsgr6sl26ZgflsOxGrnDkyQvNhHNcV1ouejIgfi2zkyfi8UeCBsZ4YHXLtoiMhFADelJw Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Domain-Id: None Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Domain-Name: None Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Identity-Status: Confirmed Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Is-Admin-Project: True Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Openstack-Nova-Api-Version: 2.92 Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Domain-Id: default Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Domain-Name: Default Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Project-Name: admin Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Role: admin,reader,member Oct 20 08:54:28.963737 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Roles: admin,reader,member Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_602743e1ebcd495bbe3a73b9f8abba4e", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}] Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Tenant: admin Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Tenant-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-Tenant-Name: admin Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User: admin Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Domain-Id: default Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Domain-Name: Default Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Id: 9804008bc9774f7b96bc93c72ddebd2f Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: X-User-Name: admin Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: method: > Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: exception: Flavor 451 could not be found. Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: action_args: {'id': '451'} Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 08:54:28.965831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:28.966844 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: ERROR nova.api.openstack.wsgi Oct 20 08:54:28.968241 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-84b042b2-eec1-4dbb-9e9e-f76bef42c1d3 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103836) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 08:54:28.969534 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-84b042b2-eec1-4dbb-9e9e-f76bef42c1d3 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.388935 Oct 20 08:54:28.969534 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 24/49] 10.222.0.36 () {60 vars in 1193 bytes} [Thu Oct 20 08:54:28 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 391 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 20 08:54:28.975370 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-b28e504c-e6d6-4fd6-aeaf-010861edaf96 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:28.990652 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-b28e504c-e6d6-4fd6-aeaf-010861edaf96 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018885 Oct 20 08:54:28.991172 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 26/50] 10.222.0.36 () {60 vars in 1210 bytes} [Thu Oct 20 08:54:28 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 08:54:29.000466 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:29.014955 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.017978 Oct 20 08:54:29.015475 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 25/51] 10.222.0.36 () {60 vars in 1210 bytes} [Thu Oct 20 08:54:28 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 08:54:29.024416 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-8fab6d03-c6ba-4532-987b-241f61226bf5 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:29.029806 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.wsgi [None req-8fab6d03-c6ba-4532-987b-241f61226bf5 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-8fab6d03-c6ba-4532-987b-241f61226bf5 admin admin] Request method failure captured: Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Accept: application/json Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Accept-Encoding: gzip, deflate Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Connection: keep-alive Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Content-Length: 0 Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Host: 10.222.0.36 Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Openstack-Api-Version: compute 2.92 Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Openstack-System-Scope: None Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: User-Agent: python-novaclient Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Auth-Token: gAAAAABjUQzEozokudtQdyINDxozNjJHlkE4oyrepjyZH05aLeLUq5ro7WeC8-7_qjkvDzc48fvc0t6fx3H93P70oSy5bdUfbqKQP8-v8-GnuExSfzlsgr6sl26ZgflsOxGrnDkyQvNhHNcV1ouejIgfi2zkyfi8UeCBsZ4YHXLtoiMhFADelJw Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Domain-Id: None Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Domain-Name: None Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Identity-Status: Confirmed Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Is-Admin-Project: True Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Openstack-Nova-Api-Version: 2.92 Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Domain-Id: default Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Domain-Name: Default Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Project-Name: admin Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Role: admin,reader,member Oct 20 08:54:29.031318 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Roles: admin,reader,member Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_602743e1ebcd495bbe3a73b9f8abba4e", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/602743e1ebcd495bbe3a73b9f8abba4e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/602743e1ebcd495bbe3a73b9f8abba4e"}]}] Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Tenant: admin Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Tenant-Id: 602743e1ebcd495bbe3a73b9f8abba4e Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-Tenant-Name: admin Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User: admin Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Domain-Id: default Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Domain-Name: Default Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Id: 9804008bc9774f7b96bc93c72ddebd2f Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: X-User-Name: admin Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: method: > Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: exception: Flavor 451 could not be found. Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: action_args: {'id': '451'} Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 08:54:29.032874 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:29.033686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: ERROR nova.api.openstack.wsgi Oct 20 08:54:29.034854 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-8fab6d03-c6ba-4532-987b-241f61226bf5 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103837) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 08:54:29.035864 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-8fab6d03-c6ba-4532-987b-241f61226bf5 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011649 Oct 20 08:54:29.035864 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 27/52] 10.222.0.36 () {60 vars in 1193 bytes} [Thu Oct 20 08:54:29 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 20 08:54:30.230167 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:30.231796 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 Oct 20 08:54:30.232185 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 26/53] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:54:30 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:30.749953 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-3e460441-1c99-4f4c-9bf0-92b8dab0260f admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:54:30.768780 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-3e460441-1c99-4f4c-9bf0-92b8dab0260f admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.401510 Oct 20 08:54:30.769345 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 28/54] 10.222.0.36 () {64 vars in 1235 bytes} [Thu Oct 20 08:54:30 2022] POST /compute/v2.1/flavors => generated 404 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:54:31.969798 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:31.970831 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001420 Oct 20 08:54:31.971221 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 27/55] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:54:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:32.492679 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-15dcfce8-ed41-46c2-89dc-6ac52fc2421d admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:54:32.510307 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-15dcfce8-ed41-46c2-89dc-6ac52fc2421d admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.404926 Oct 20 08:54:32.510881 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 29/56] 10.222.0.36 () {64 vars in 1235 bytes} [Thu Oct 20 08:54:32 2022] POST /compute/v2.1/flavors => generated 405 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 20 08:54:33.721644 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:33.722690 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001419 Oct 20 08:54:33.723088 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 28/57] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:54:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:34.248404 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-586f5a52-862b-47d3-a438-692e80f871d3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:54:34.267241 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-586f5a52-862b-47d3-a438-692e80f871d3 admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.406623 Oct 20 08:54:34.267720 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 30/58] 10.222.0.36 () {64 vars in 1235 bytes} [Thu Oct 20 08:54:33 2022] POST /compute/v2.1/flavors => generated 407 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:54:35.465999 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:35.467042 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002043 Oct 20 08:54:35.467432 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 29/59] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:54:35 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:35.983039 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-280fc912-be5b-4caa-860f-3107691519fb admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 08:54:36.002544 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-280fc912-be5b-4caa-860f-3107691519fb admin admin] 10.222.0.36 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.401284 Oct 20 08:54:36.003094 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 31/60] 10.222.0.36 () {64 vars in 1235 bytes} [Thu Oct 20 08:54:35 2022] POST /compute/v2.1/flavors => generated 409 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:54:37.223331 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:37.224351 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001489 Oct 20 08:54:37.224746 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 30/61] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:54:37 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:54:37.740714 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-521d325a-0264-43be-bd2f-aa4527d1a3d0 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:54:37.761165 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-521d325a-0264-43be-bd2f-aa4527d1a3d0 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.399378 Oct 20 08:54:37.761695 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 32/62] 10.222.0.36 () {60 vars in 1202 bytes} [Thu Oct 20 08:54:37 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 08:55:33.812387 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:55:33.813343 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-cdeb9a0d-2974-4862-9b37-d646300b9472 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001443 Oct 20 08:55:33.813744 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 31/63] 10.222.0.36 () {54 vars in 931 bytes} [Thu Oct 20 08:55:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:55:33.824866 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:55:34.204891 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-dbb065a1-c246-4315-973d-e2d61c6012f2 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:55:34.227820 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-dbb065a1-c246-4315-973d-e2d61c6012f2 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.407778 Oct 20 08:55:34.228418 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 33/64] 10.222.0.36 () {60 vars in 1292 bytes} [Thu Oct 20 08:55:33 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 08:55:34.250800 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writing DB stats nova_api:SELECT=14 {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:55:34.254778 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-acac6cf5-3936-4b54-9b38-5032538401d9 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:55:34.261909 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-acac6cf5-3936-4b54-9b38-5032538401d9 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012542 Oct 20 08:55:34.262426 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 32/65] 10.222.0.36 () {60 vars in 1283 bytes} [Thu Oct 20 08:55:34 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:55:34.273997 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-3d0b371a-01eb-4ebc-a5e8-9667f6f6ea27 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:55:34.281601 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-3d0b371a-01eb-4ebc-a5e8-9667f6f6ea27 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011119 Oct 20 08:55:34.282103 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 34/66] 10.222.0.36 () {60 vars in 1286 bytes} [Thu Oct 20 08:55:34 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:55:34.293717 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-c7e8b079-a307-4f20-bc27-9498f0e24ac0 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:55:34.300952 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-c7e8b079-a307-4f20-bc27-9498f0e24ac0 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011400 Oct 20 08:55:34.301480 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 33/67] 10.222.0.36 () {60 vars in 1286 bytes} [Thu Oct 20 08:55:34 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:55:34.312681 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-72eed460-ca1e-4508-9967-0e6e899dc3f3 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:55:34.319681 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-72eed460-ca1e-4508-9967-0e6e899dc3f3 admin admin] 10.222.0.36 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010474 Oct 20 08:55:34.320184 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 35/68] 10.222.0.36 () {60 vars in 1283 bytes} [Thu Oct 20 08:55:34 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 11 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 08:58:33.174033 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-c7e8b079-a307-4f20-bc27-9498f0e24ac0 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:58:33.175071 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-c7e8b079-a307-4f20-bc27-9498f0e24ac0 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Oct 20 08:58:33.175460 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 34/69] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:58:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:58:33.316498 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writing DB stats nova_api:SELECT=6 {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:58:33.473967 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-4d5ee9a9-0335-4165-b764-60dd3c212fd9 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:58:33.478622 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-4d5ee9a9-0335-4165-b764-60dd3c212fd9 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:58:33.479694 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-4d5ee9a9-0335-4165-b764-60dd3c212fd9 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:58:33.480118 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-4d5ee9a9-0335-4165-b764-60dd3c212fd9 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:58:33.480817 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-4d5ee9a9-0335-4165-b764-60dd3c212fd9 admin admin] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:58:33.481254 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-4d5ee9a9-0335-4165-b764-60dd3c212fd9 admin admin] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:58:33.481682 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-4d5ee9a9-0335-4165-b764-60dd3c212fd9 admin admin] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:58:33.512129 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-4d5ee9a9-0335-4165-b764-60dd3c212fd9 admin admin] 10.222.0.36 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.196849 Oct 20 08:58:33.512672 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 36/70] 10.222.0.36 () {60 vars in 1193 bytes} [Thu Oct 20 08:58:33 2022] GET /compute/v2.1/os-services => generated 791 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:58:34.585415 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-c7e8b079-a307-4f20-bc27-9498f0e24ac0 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:58:34.586418 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-c7e8b079-a307-4f20-bc27-9498f0e24ac0 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001423 Oct 20 08:58:34.586854 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 35/71] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:58:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:58:34.774148 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-feccd90a-bff9-4446-ac42-887ab025b1a9 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:58:34.778182 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-feccd90a-bff9-4446-ac42-887ab025b1a9 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:58:34.778683 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-feccd90a-bff9-4446-ac42-887ab025b1a9 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:58:34.779106 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-feccd90a-bff9-4446-ac42-887ab025b1a9 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:58:34.779836 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-feccd90a-bff9-4446-ac42-887ab025b1a9 admin admin] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:58:34.780277 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-feccd90a-bff9-4446-ac42-887ab025b1a9 admin admin] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:58:34.780686 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-feccd90a-bff9-4446-ac42-887ab025b1a9 admin admin] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:58:34.800085 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-feccd90a-bff9-4446-ac42-887ab025b1a9 admin admin] 10.222.0.36 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.073972 Oct 20 08:58:34.800558 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 37/72] 10.222.0.36 () {60 vars in 1193 bytes} [Thu Oct 20 08:58:34 2022] GET /compute/v2.1/os-services => generated 791 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 08:58:46.289163 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-c7e8b079-a307-4f20-bc27-9498f0e24ac0 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:58:46.290182 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-c7e8b079-a307-4f20-bc27-9498f0e24ac0 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001421 Oct 20 08:58:46.290609 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 36/73] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:58:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:58:46.434013 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writing DB stats nova_cell0:SELECT=4 {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:58:46.434738 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writing DB stats nova_cell1:SELECT=4 {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:58:46.435374 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG dbcounter [-] [103837] Writing DB stats nova_api:SELECT=8 {{(pid=103837) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:58:46.483446 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-b36abe0b-72df-4f05-8c47-168fe9d73403 admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:58:46.486896 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-b36abe0b-72df-4f05-8c47-168fe9d73403 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:58:46.487366 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-b36abe0b-72df-4f05-8c47-168fe9d73403 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:58:46.487795 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-b36abe0b-72df-4f05-8c47-168fe9d73403 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:58:46.488509 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-b36abe0b-72df-4f05-8c47-168fe9d73403 admin admin] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:58:46.488952 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-b36abe0b-72df-4f05-8c47-168fe9d73403 admin admin] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:58:46.489367 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-b36abe0b-72df-4f05-8c47-168fe9d73403 admin admin] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:58:46.514432 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-b36abe0b-72df-4f05-8c47-168fe9d73403 admin admin] 10.222.0.36 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.085327 Oct 20 08:58:46.514936 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 38/74] 10.222.0.36 () {60 vars in 1193 bytes} [Thu Oct 20 08:58:46 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 08:58:47.584403 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG nova.api.openstack.wsgi [None req-c7e8b079-a307-4f20-bc27-9498f0e24ac0 admin admin] Calling method '>' {{(pid=103836) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:58:47.585478 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: INFO nova.api.openstack.requestlog [None req-c7e8b079-a307-4f20-bc27-9498f0e24ac0 admin admin] 10.222.0.36 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001453 Oct 20 08:58:47.585877 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: [pid: 103836|app: 0|req: 37/75] 10.222.0.36 () {56 vars in 1074 bytes} [Thu Oct 20 08:58:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 08:58:47.777742 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG nova.api.openstack.wsgi [None req-251d27a5-fa72-4444-8eb0-4a3dbeea592d admin admin] Calling method '>' {{(pid=103837) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 08:58:47.781095 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-251d27a5-fa72-4444-8eb0-4a3dbeea592d admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:58:47.781567 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-251d27a5-fa72-4444-8eb0-4a3dbeea592d admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:58:47.782015 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-251d27a5-fa72-4444-8eb0-4a3dbeea592d admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:58:47.782730 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-251d27a5-fa72-4444-8eb0-4a3dbeea592d admin admin] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:58:47.783174 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-251d27a5-fa72-4444-8eb0-4a3dbeea592d admin admin] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:58:47.783584 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: DEBUG oslo_concurrency.lockutils [None req-251d27a5-fa72-4444-8eb0-4a3dbeea592d admin admin] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103837) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:58:47.804370 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: INFO nova.api.openstack.requestlog [None req-251d27a5-fa72-4444-8eb0-4a3dbeea592d admin admin] 10.222.0.36 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.077281 Oct 20 08:58:47.804905 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: [pid: 103837|app: 0|req: 39/76] 10.222.0.36 () {60 vars in 1193 bytes} [Thu Oct 20 08:58:47 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 08:59:44.554110 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 20 08:59:44.554904 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: Thu Oct 20 08:59:44 2022 - graceful shutdown triggered... Oct 20 08:59:44.555396 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: Gracefully killing worker 1 (pid: 103836)... Oct 20 08:59:44.555771 dv-cinder-master-3-iscsi devstack@n-api.service[103837]: Gracefully killing worker 2 (pid: 103837)... Oct 20 08:59:44.680206 dv-cinder-master-3-iscsi devstack@n-api.service[103836]: DEBUG dbcounter [-] [103836] Writing DB stats nova_api:SELECT=4 {{(pid=103836) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 08:59:45.556128 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: worker 1 buried after 1 seconds Oct 20 08:59:45.556128 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: worker 2 buried after 1 seconds Oct 20 08:59:45.556128 dv-cinder-master-3-iscsi devstack@n-api.service[103835]: goodbye to uWSGI. Oct 20 08:59:45.558175 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 20 08:59:45.558746 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 20 08:59:45.561817 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 20 08:59:45.570763 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 20 08:59:45.572658 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 20 08:59:45 2022] *** Oct 20 08:59:45.572658 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 20 08:59:45.572658 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 20 08:59:45.572658 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: nodename: dv-cinder-master-3-iscsi Oct 20 08:59:45.572658 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: machine: x86_64 Oct 20 08:59:45.572658 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: clock source: unix Oct 20 08:59:45.572658 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: pcre jit disabled Oct 20 08:59:45.572658 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: detected number of CPU cores: 4 Oct 20 08:59:45.572658 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: current working directory: / Oct 20 08:59:45.572658 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: detected binary path: /usr/bin/uwsgi-core Oct 20 08:59:45.574071 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: your processes number limit is 47680 Oct 20 08:59:45.574071 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: your memory page size is 4096 bytes Oct 20 08:59:45.574071 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: detected max file descriptor number: 2048 Oct 20 08:59:45.574071 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: lock engine: pthread robust mutexes Oct 20 08:59:45.574071 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: thunder lock: enabled Oct 20 08:59:45.574071 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 20 08:59:45.574071 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 20 08:59:45.614788 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: Python main interpreter initialized at 0x55b86d0529c0 Oct 20 08:59:45.614788 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: python threads support enabled Oct 20 08:59:45.614788 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: your server socket listen backlog is limited to 100 connections Oct 20 08:59:45.614788 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: your mercy for graceful operations on workers is 90 seconds Oct 20 08:59:45.615257 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: mapped 403077 bytes (393 KB) for 2 cores Oct 20 08:59:45.615257 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: *** Operational MODE: preforking *** Oct 20 08:59:45.615527 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: *** uWSGI is running in multiple interpreter mode *** Oct 20 08:59:45.615527 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: spawned uWSGI master process (pid: 119046) Oct 20 08:59:45.615735 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 20 08:59:45.618509 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: spawned uWSGI worker 1 (pid: 119067, cores: 1) Oct 20 08:59:45.618509 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: spawned uWSGI worker 2 (pid: 119068, cores: 1) Oct 20 08:59:45.618509 dv-cinder-master-3-iscsi devstack@n-api.service[119046]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 20 08:59:46.273811 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 20 08:59:46.279822 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 20 08:59:47.424685 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 08:59:47.424685 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: from cryptography.utils import int_from_bytes Oct 20 08:59:47.425303 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 08:59:47.425303 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: from cryptography.utils import int_from_bytes Oct 20 08:59:47.440733 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 08:59:47.440733 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: from cryptography.utils import int_from_bytes Oct 20 08:59:47.441255 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 08:59:47.441255 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: from cryptography.utils import int_from_bytes Oct 20 08:59:49.817151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 20 08:59:49.817698 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 20 08:59:49.817993 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 20 08:59:49.818219 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 20 08:59:49.818575 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 20 08:59:49.818949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.819274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.819599 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.820087 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.820421 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.820736 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.821065 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.821391 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.821723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.821986 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.822306 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.822670 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.823025 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.823336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.823650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.823976 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.824281 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.824607 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.824912 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.825414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.826947 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.826947 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.826947 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.826947 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.827304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.827529 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.827719 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.828035 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.828352 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.828658 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.829062 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.829372 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.829693 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.830001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.830400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.830759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.831039 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.831365 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.831680 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.831976 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.832298 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.832613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.832927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.833236 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.833544 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.833840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.834168 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.834463 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.834791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.835096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.835343 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.835651 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.835967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.836281 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.836596 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.836844 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.837088 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.837404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.837734 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.838027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.838345 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.839189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.839498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.840055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.840359 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.840662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.840970 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.841279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.841591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.841907 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.842208 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.842545 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.842944 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.36 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.843189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.36 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.843489 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.843831 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.844145 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.844449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.844773 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.845087 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.845393 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.845696 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.846047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.846369 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.846723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.847027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.847276 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.847580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.847885 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.848201 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.848507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.848813 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.849116 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.849414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.849716 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.850022 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.850335 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.850650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.850911 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 20 08:59:49.851427 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.851590 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 20 08:59:49.851748 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.851748 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.852044 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 20 08:59:49.852201 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.852349 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 20 08:59:49.852502 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.852658 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 20 08:59:49.852806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.852973 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.853131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.853287 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.853441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.853441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.853739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.853892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.854048 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.854930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.855090 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.855236 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.855385 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.855540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.855540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.855850 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.856026 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.856202 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.856377 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.856539 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.856685 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.856838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.856992 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.857150 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.857296 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.857458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.857629 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.857787 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.857934 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.858086 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.858240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.858401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.858595 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.858753 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.858907 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.859054 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.859216 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.859885 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.860050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.860210 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.860375 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.860375 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.860929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.861098 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.861840 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.861999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.862147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.862308 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.862463 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.862702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.862871 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.863018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.863164 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.863407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.863558 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.863718 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.863865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.864019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.864177 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.864383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.864545 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.864692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.864847 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.865015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.865175 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.865322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.865468 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.865622 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.865769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.865923 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.866084 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.866238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.866238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.866554 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.866554 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.866855 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.867025 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.867180 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.867327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.867479 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.867632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.867791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.867940 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.868095 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.868242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.868395 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.868549 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.868709 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.868709 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.869009 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.869156 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.869301 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.869456 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.869603 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.869791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.870044 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.870203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.870351 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.870540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.870872 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.871028 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.871194 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.871359 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.872081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.872081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.872398 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.872554 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.872701 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.872854 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.872999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.873160 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.873314 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.873517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.873683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.873838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.873838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.874144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.874302 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.874518 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.874724 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.874892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.875047 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.875200 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.875355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.875509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.875674 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.875829 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.875977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.876129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.876278 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.876429 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.876582 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.876737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.876892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.877046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.877200 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.877348 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.877493 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.877670 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.877822 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.877967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.878120 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.878275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.878429 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.878646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.878802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.878949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.879095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.36 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.879248 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.879404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.36 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.879569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.879723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.879878 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.880024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.880169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.880324 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.880472 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.880619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.880790 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.881100 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.881260 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.881422 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.881570 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.882335 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.882512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.882678 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.882827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.882983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.883130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.883275 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.883429 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.883575 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.883732 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.883879 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.884024 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.884177 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.884346 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.884574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.884743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.884899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.885053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.885216 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.885422 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.885579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.885743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.885902 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.886057 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.886205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.886350 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.886556 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.886720 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.886871 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.887025 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.887179 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.887325 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.887482 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.887643 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.887802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.887950 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.888095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.888255 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.888411 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.888579 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.888739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.888896 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.889051 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.889208 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.889356 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.889514 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.889661 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.889814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.889969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.890117 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.890270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.890436 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.890622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.890622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.890918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.891325 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.891489 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.891645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.891792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.892502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.892675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.892826 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.892980 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.893136 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.893303 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.893458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.893614 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.893761 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.893915 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.894062 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.894217 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.894364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.894657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.894813 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.894968 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.895121 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.895269 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.895438 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.895593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.895749 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.895895 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.896080 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.896231 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.896394 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.896543 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.896690 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.896835 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.896981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.897275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.897481 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.897653 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.897804 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.897950 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.898105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.898261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.898424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.898608 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.898764 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.898911 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.899057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.899212 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.899369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.899534 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.899689 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.899843 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.899989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.900144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.900291 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.900438 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.900584 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.900729 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.900881 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.901690 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.901847 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 08:59:49.902021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.902701 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.902871 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.903056 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.903217 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.903366 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.903512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.903659 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.903805 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.903967 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.904117 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.904270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.904461 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.904621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.904831 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.904989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.905144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.905290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.905444 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.905590 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.905745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.905901 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.906049 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.906194 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.906340 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.906531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.906714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.906863 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.907015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.907161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.907315 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.907469 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.907643 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.907793 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.907947 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.908095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.908248 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.908394 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.908541 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.908541 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.908708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.908790 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.908876 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.908964 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909063 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909158 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909246 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909338 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909425 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909610 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909705 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909792 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.909983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.910083 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.910171 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.910256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.910349 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.910442 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.910565 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.910664 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.910752 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.910838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.910923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.911008 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.911094 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.911188 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.911275 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.911360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.911453 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.911546 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.911648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.911903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.912005 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.912094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.912180 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.912288 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.912913 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913187 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913534 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.913621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914083 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914169 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914169 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914169 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914429 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914542 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914542 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914863 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914863 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.914863 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.915109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.915203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.915203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.915203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.915203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.915203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.915203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.915651 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.915743 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.915835 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.916040 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.916354 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.916447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.917264 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.917745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.917853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.917853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.918189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.918320 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.918463 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.918758 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.918939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.919099 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.919370 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.919471 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.919712 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.919980 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.920085 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.920325 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.920579 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.920723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.920940 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.921170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.921278 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.921566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.921770 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.922028 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.922201 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.922384 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.923578 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.923686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.923791 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.923888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.923977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.924078 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.924174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.924174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.924174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.924435 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.924550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.924699 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.925365 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.925463 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.925562 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.925652 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.925746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.925888 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.926192 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.926294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.926550 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.926827 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.926922 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.927132 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.927410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.927512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.927750 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.928031 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.928134 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.928344 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.928623 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.928726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.928982 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.929211 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.929313 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.929618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.929796 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.929953 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.930274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.930365 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.930627 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.930944 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.931038 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.931264 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.931531 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.931632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.931875 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.932136 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.932289 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.932539 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.932979 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.933100 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.933207 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.933303 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.933473 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.933786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.933886 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.934111 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.934395 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.934529 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.934760 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.935026 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.935133 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.935370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.935630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.935731 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.936036 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.936230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.936354 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.936676 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.936823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.936987 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.937301 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.937400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.937646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.937908 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.938017 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.938218 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.938549 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.938658 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.938870 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.939174 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.939275 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.939489 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.939765 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.939874 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.940110 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.940395 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.940497 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.940722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.940998 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.941091 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.941326 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.941583 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.941683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.941973 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.942259 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.942371 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.942605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.942759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.943191 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.943306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.943524 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.943670 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.943953 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.944126 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.944263 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.944434 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.944745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.945061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.945217 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.945370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.945686 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.945818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.946049 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.946303 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.946403 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.946650 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.946952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.947049 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.947254 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.947566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.947667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.947917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.948194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.948297 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.948558 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.948817 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.948929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.949173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.949429 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.949532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.949790 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.950020 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.950114 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.950393 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.950615 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.950723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.951021 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.951204 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.951367 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.951680 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.951774 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.951987 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.952370 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.952482 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.952614 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.952860 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.952978 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.953397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.953500 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.953604 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.953824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.954105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.954225 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.954450 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.954617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.954799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.955108 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.955230 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.955418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.955699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.955794 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.956034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.956275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.956385 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.956654 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.956868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.956961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.957258 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.957449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.957564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.957875 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.958045 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.958191 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.958932 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.959038 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.959135 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.959233 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.959416 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.959714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.959816 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.960067 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.960446 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.960568 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.960704 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.960922 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.961071 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.961392 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.961494 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.961698 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.962002 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.962095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.962307 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.962612 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.962728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.962956 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.964042 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.964376 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.964497 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.964843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.964843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.965011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.965158 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.965257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.965530 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.965745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.965840 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.966126 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.966362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.966456 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.966774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.966979 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.967074 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.967378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.967566 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.967701 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.968011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.968181 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.968315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.968620 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.968748 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.968926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.969258 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.969352 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.969564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.969835 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.969938 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.970219 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.970426 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.970585 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.970875 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.971029 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.971180 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.971503 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.971596 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.971808 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.972134 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.972233 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.973064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.973903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.974034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.974145 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.974235 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.974235 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.974614 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.974724 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.974813 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.974899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.974985 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.975070 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.975315 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.975415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.975511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.975598 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.975694 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.975789 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.975887 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.975983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.976078 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.976174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.976272 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.976370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.976457 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.976543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.976628 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.976721 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.976819 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.977034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.977132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.977344 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.977446 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.977650 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.977759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.977963 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.978066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.978268 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.978370 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.978608 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.978707 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.978918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.979014 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.979799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.979914 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.980111 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.980215 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.980424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.980518 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.980736 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.980838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.981040 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.981143 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.981369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.981506 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.981673 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.981767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.981980 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.982092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.982283 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.982375 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.982622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.982735 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.982925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.983071 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.983229 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.983311 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.983539 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.983630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.983862 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.983946 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.984230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.984335 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.984584 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.984689 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.984774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.984848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.985071 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.985170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.985378 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.985469 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.985752 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.985867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.985964 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.986049 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.986275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.986383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.986598 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.986692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.986905 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.987209 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.987845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.987845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.988119 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.990539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.990886 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.990886 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991058 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991153 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991338 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991432 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991673 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991755 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.991936 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992103 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992194 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992298 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992390 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992567 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992658 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992744 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992836 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.992928 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993013 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993188 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993279 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993372 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993541 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993727 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993819 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.993998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.994137 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.994270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.994814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.994972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.995065 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.995142 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.995235 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.996140 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.996314 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.996472 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.996635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.996791 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.996945 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.997106 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.997294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.997616 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.997786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.997969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.998125 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.998288 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.998451 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.998643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.998807 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.998971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.999126 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.999292 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.999461 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.999627 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.999788 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:49.999942 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.000119 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.000284 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.000446 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.000607 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.000770 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.000925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.001079 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.001246 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.001400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.001554 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.001804 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.001975 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.002150 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.002314 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.002621 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.002787 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.002954 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.003146 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.003384 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.003556 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.003712 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.003878 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.004040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.004207 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.004371 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.004371 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.004726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.004881 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.005043 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.005205 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.005392 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.006075 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.006250 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.006416 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.006597 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.006723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.006828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.006918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007014 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007101 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007195 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007387 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007571 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007664 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007750 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007846 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.007940 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008039 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008126 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008217 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008402 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008489 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008667 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008758 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.008944 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.009045 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.009137 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.009223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.009324 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.009412 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.009553 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.009648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.009735 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.009827 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.009913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.010011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.010104 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.010190 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.010441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.010589 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.010784 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.010901 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.011097 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.011198 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.011500 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.011603 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.011815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.011916 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.012132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.012237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.012453 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.012555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.012779 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.012880 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.013087 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.013187 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.013404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.013517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.013711 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.013811 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.014027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.014119 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.014349 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.014453 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.014689 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.014792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.015004 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.015106 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.015312 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.015413 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.015637 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.015726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.015960 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.016050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.016281 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.016381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.016600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.016700 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.016927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.017027 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.017321 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.017414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.017632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.017725 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.017955 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.018048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.018286 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.018494 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.018622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.018774 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.018929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.019052 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.019239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.019339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.019543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.019634 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.019852 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.019954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.020168 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.020261 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.020476 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.020581 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.020789 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.020906 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.021104 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.021211 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.021413 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.021507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.021717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.021825 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.022023 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.022116 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.022329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.022429 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.022679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.022776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.022976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.023071 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.023292 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.023393 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.023610 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.023710 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.023929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.024021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.024235 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.024329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.024548 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.024649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.024856 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.024948 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.025161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.025254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.025467 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.025560 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.025769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.025856 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.026626 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.026742 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.026981 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.027082 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.027292 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.027392 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.027617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.027718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.027953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.028046 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.028277 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.028368 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.028597 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.028697 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.028933 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.029030 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.029258 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.029359 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.029575 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.029676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.029892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.029999 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.030218 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.030319 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.030545 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.030671 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.030853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.030946 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.031161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.031253 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.031465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.031567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.031771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.031870 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.032074 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.032166 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.032381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.032475 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.032681 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.032772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.032985 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.033091 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.033287 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.033385 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.033589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.033702 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.033900 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.034005 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.034204 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.034303 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.034540 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.034648 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.034848 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.034954 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.035154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.035255 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.035458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.035557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.035783 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.035887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.036095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.036187 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.036423 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.036516 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.036736 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.036829 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.037048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.037147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.037361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.037453 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.037677 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.037771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.037992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.038084 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.038306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.038399 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.038642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.038746 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.038955 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.039048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.039269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.039395 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.039576 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.039720 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.039889 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.040034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.040200 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.040352 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.040517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.040669 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.040990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.041090 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.041401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.041558 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.042270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.042459 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.042614 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.042818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.042920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.043143 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.043238 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.043467 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.043578 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.043792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.043885 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.044116 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.044221 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.044434 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.044533 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.044746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.044848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.045062 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.045152 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.045380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.045481 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.045702 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.045807 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.046030 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.046119 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.046351 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.046441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.046715 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.046821 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.047035 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.047127 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.047355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.047439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.047675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.047769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.047996 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.048097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.048310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.048407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.048624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.048725 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.048940 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.049045 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.049257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.049364 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.049569 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.049661 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.049877 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.049977 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.050188 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.050294 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.050518 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.050615 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.050832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.050934 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.051142 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.051248 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.051455 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.051548 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.051772 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.052096 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.052423 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.052517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.052740 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.052832 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.053051 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.053152 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.053364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.053458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.053685 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.053818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.054064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.054203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.054383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.054538 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.054730 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.054874 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.055054 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.055198 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.055522 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.055621 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.055842 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.055934 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.056159 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.056256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.056541 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.056633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.056858 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.056949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.057161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.057271 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.057528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.057631 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.057839 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.057962 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.058143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.058279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.058493 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.058681 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.058797 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.059001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.059110 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.059319 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.059419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.059630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.059725 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.059959 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.060227 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.060330 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.060572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.060664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.060891 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.060990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.061205 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.061298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.061539 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.061741 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.061861 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.062092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.062188 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.062416 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.062525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.062771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.062866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.063097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.063197 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.063415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.063516 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.063726 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.063818 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.064032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.064124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.064337 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.064437 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.064648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.064748 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.064961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.065062 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.065265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.065358 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.065572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.065673 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.065875 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.065967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.066187 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.066275 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.066514 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.066626 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.066822 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.066929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.067128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.067257 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.067432 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.067535 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.067733 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.067831 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.068037 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.068131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.068341 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.068447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.068644 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.068737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.068955 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.069051 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.069259 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.069353 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.069564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.069663 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.069865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.069958 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.070169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.070262 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.070521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.070619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.070810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.070904 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.071123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.071220 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.071431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.071524 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.071742 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.071841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.072048 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.072149 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.072365 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.072465 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.072676 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.072773 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.073023 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.073108 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.073328 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.073428 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.073636 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.073742 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.073954 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.074055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.074268 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.074369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.074615 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.074832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.074927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.075165 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.075264 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.075490 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.075590 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.075823 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.075917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.076156 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.076404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.076494 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.076742 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.076846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.077045 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.077132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.077369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.077504 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.077679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.077774 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.077992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.078087 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.078303 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.078397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.078633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.078729 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.078944 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.079048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.079253 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.079353 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.079562 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.079657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.079872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.079972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.080181 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.080286 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.080490 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.080584 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.080798 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.080898 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.081103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.081209 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.081411 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.081517 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.081723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.081853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.082040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.082157 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.082353 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.082499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.082687 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.082785 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.083000 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.083095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.083311 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.083412 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.083627 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.083731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.083937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.084042 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.084248 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.084342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.084559 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.084660 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.084871 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.084966 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.085188 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.085289 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.085502 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.085605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.085813 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.085914 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.086116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.086209 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.086426 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.086551 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.086771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.086869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.087105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.087246 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.087423 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.087565 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.087731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.087825 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.088071 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.088170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.088395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.089011 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.089227 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.089335 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.089805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.089904 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.090118 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.090213 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.090449 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.090605 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.090799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.090899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.091115 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.091211 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.091441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.091543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.091747 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.091913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.092059 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.092190 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.092365 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.092459 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.092675 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.092782 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.092993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.093089 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.093305 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.093397 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.093609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.093711 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.093913 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.094005 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.094221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.094321 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.094543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.094653 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.094850 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.094943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.095155 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.095255 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.095458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.095553 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.095765 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.095858 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.096065 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.096167 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.096368 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.096469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.096670 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.096812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.096971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.097066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.097274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.097368 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.097575 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.097709 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.097879 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.097976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.098178 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.098271 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.098490 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.098585 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.098802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.098913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.099105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.099198 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.099405 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.099497 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.099708 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.099809 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.100014 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.100115 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.100320 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.100414 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.100629 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.100732 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.100933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.101033 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.101265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.101357 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.101576 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.101682 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.101897 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.102000 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.102218 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.102311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.102553 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.102650 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.102869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.103010 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.103181 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.103274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.103503 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.103603 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.103816 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.103909 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.104216 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.104848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.105042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.105145 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.105357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.105458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.105669 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.105775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.105986 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.106086 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.106303 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.106397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.106633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.106744 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.106947 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.107042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.107264 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.107357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.107583 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.107724 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.107901 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.108004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.108208 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.108307 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.108536 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.108669 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.108848 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.108943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.109156 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.109250 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.109466 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.109560 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.109774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.109867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.110084 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.110177 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.110398 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.110511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.110733 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.110827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.111043 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.111145 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.111357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.111483 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.111669 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.111769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.111988 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.112095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.112300 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.112401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.112611 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.112715 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.112930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.113036 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.113249 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.113345 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.113577 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.113680 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.113893 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.114010 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.114208 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.114301 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.114559 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.114668 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.114891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.114986 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.115202 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.115295 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.115506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.115607 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.115748 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=119067) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 20 08:59:50.115875 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.116207 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.116527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.116842 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.117152 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 08:59:50.117399 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=119068) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 20 08:59:50.177706 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO dbcounter [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Registered counter for database nova_api Oct 20 08:59:50.180168 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO dbcounter [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Registered counter for database nova_api Oct 20 08:59:50.199297 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_db.sqlalchemy.engines [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119067) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:59:50.201275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_db.sqlalchemy.engines [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119068) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:59:50.206380 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writer thread running {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:59:50.207921 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writer thread running {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:59:50.263952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.context [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dd273305-b4a6-40de-9d39-cde54b55a76f(cell1) {{(pid=119068) load_cells /opt/stack/nova/nova/context.py:464}} Oct 20 08:59:50.264189 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.context [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dd273305-b4a6-40de-9d39-cde54b55a76f(cell1) {{(pid=119067) load_cells /opt/stack/nova/nova/context.py:464}} Oct 20 08:59:50.267966 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:59:50.268193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:59:50.268435 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:59:50.268683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:59:50.270024 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:59:50.270417 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:59:50.270809 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:59:50.271186 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 08:59:50.271300 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:59:50.271636 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 08:59:50.273202 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:59:50.273616 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 08:59:50.275197 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO dbcounter [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Registered counter for database nova_cell0 Oct 20 08:59:50.275621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO dbcounter [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Registered counter for database nova_cell0 Oct 20 08:59:50.279770 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO dbcounter [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Registered counter for database nova_cell1 Oct 20 08:59:50.280204 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO dbcounter [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Registered counter for database nova_cell1 Oct 20 08:59:50.286989 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_db.sqlalchemy.engines [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119068) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:59:50.289324 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_db.sqlalchemy.engines [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119068) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:59:50.290879 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_db.sqlalchemy.engines [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119067) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:59:50.292127 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_db.sqlalchemy.engines [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119067) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:59:50.293377 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writer thread running {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:59:50.299184 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writer thread running {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:59:50.300514 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writer thread running {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:59:50.438064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writer thread running {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:59:50.449780 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO dbcounter [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Registered counter for database nova_cell0 Oct 20 08:59:50.451449 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO dbcounter [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Registered counter for database nova_cell0 Oct 20 08:59:50.457347 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_db.sqlalchemy.engines [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119068) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:59:50.458988 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_db.sqlalchemy.engines [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119067) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 08:59:50.461940 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writer thread running {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:59:50.463691 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writer thread running {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 08:59:50.776560 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: WARNING keystonemiddleware.auth_token [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 20 08:59:50.780682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: WARNING keystonemiddleware.auth_token [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 20 08:59:50.783903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: WARNING keystonemiddleware.auth_token [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 20 08:59:50.787994 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: WARNING keystonemiddleware.auth_token [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 20 08:59:51.128999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: WARNING keystonemiddleware.auth_token [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 20 08:59:51.134317 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: WARNING keystonemiddleware.auth_token [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 20 08:59:51.136448 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: WARNING keystonemiddleware.auth_token [None req-f7910160-5ba1-4e07-9ec3-bc75830e9d7e None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 20 08:59:51.142335 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b86d0529c0 pid: 119068 (default app) Oct 20 08:59:51.142811 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: WARNING keystonemiddleware.auth_token [None req-3aa53c49-c5a6-4dc9-b1d4-97cd9c93b691 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 20 08:59:51.150001 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b86d0529c0 pid: 119067 (default app) Oct 20 09:00:26.619019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=3 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:00:26.619978 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=3 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:00:26.621015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=3 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:00:26.621659 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=3 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:00:26.740615 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=3 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:00:26.742317 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=3 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:00:26.743316 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=3 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:00:26.744354 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=3 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:00:28.425076 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1ff4da23-42fc-4a2a-9a3d-f8c5156338fe tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Action: 'create', calling method: .version_select at 0x7f5afc9b4b80>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-293237158"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:00:28.548029 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8ec15b2e-4914-4da9-a9ba-6a1b27945308 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] Action: 'create', calling method: .version_select at 0x7f5afc9b4b80>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1099586399", "metadata": {"Type": "work"}}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:00:30.325769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8ec15b2e-4914-4da9-a9ba-6a1b27945308 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] 10.222.0.36 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 3.599493 Oct 20 09:00:30.326498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 1/1] 10.222.0.36 () {60 vars in 1156 bytes} [Thu Oct 20 09:00:26 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 3602 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 20 09:00:30.343309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e3b125e6-aa5b-445d-830a-f944707d8315 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] Calling method '.version_select at 0x7f5afc88d9d0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:30.552533 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1ff4da23-42fc-4a2a-9a3d-f8c5156338fe tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 3.944865 Oct 20 09:00:30.553162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 1/2] 10.222.0.36 () {60 vars in 1155 bytes} [Thu Oct 20 09:00:26 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 3947 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 20 09:00:30.569955 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0ab89a01-9d4d-46f0-b75f-62dba31cd164 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc8909d0>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:30.623054 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e3b125e6-aa5b-445d-830a-f944707d8315 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960" status: 200 len: 353 microversion: 2.1 time: 0.284554 Oct 20 09:00:30.624048 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:00:30 2022] GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960 => generated 353 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:31.235800 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0ab89a01-9d4d-46f0-b75f-62dba31cd164 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b" status: 200 len: 344 microversion: 2.1 time: 0.670306 Oct 20 09:00:31.237040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:00:30 2022] GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b => generated 344 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:00:31.637091 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c6126d8e-8747-4ca6-a708-2a14bf7d48a1 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] Calling method '.version_select at 0x7f5afc81e0d0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:31.706161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c6126d8e-8747-4ca6-a708-2a14bf7d48a1 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960" status: 200 len: 354 microversion: 2.1 time: 0.072793 Oct 20 09:00:31.717148 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 3/5] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:00:31 2022] GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960 => generated 354 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:31.720539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f38b6709-933d-4b00-8804-ea5bb1b99919 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] Calling method '.version_select at 0x7f5afc8200d0>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:31.789561 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f38b6709-933d-4b00-8804-ea5bb1b99919 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960" status: 200 len: 354 microversion: 2.1 time: 0.073098 Oct 20 09:00:31.790916 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 3/6] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:00:31 2022] GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960 => generated 354 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:00:31.814214 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8cd90c45-c85e-4eb5-b7ae-bd78335ae7f6 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] Calling method '.version_select at 0x7f5afc81e790>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:31.947629 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8cd90c45-c85e-4eb5-b7ae-bd78335ae7f6 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] 10.222.0.36 "DELETE /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960" status: 202 len: 0 microversion: 2.1 time: 0.141784 Oct 20 09:00:31.959023 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1248 bytes} [Thu Oct 20 09:00:31 2022] DELETE /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960 => generated 0 bytes in 154 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:00:31.959556 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-95d28c55-756b-4aaa-b925-35fd9fff6810 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] Calling method '.version_select at 0x7f5afc820820>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:32.063722 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-95d28c55-756b-4aaa-b925-35fd9fff6810 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960" status: 200 len: 353 microversion: 2.1 time: 0.110348 Oct 20 09:00:32.072412 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:00:31 2022] GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960 => generated 353 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:00:32.252223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-145004f8-c1cb-48a1-bfeb-9edbabd2d42b tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc88daf0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:32.583974 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-145004f8-c1cb-48a1-bfeb-9edbabd2d42b tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b" status: 200 len: 345 microversion: 2.1 time: 0.337120 Oct 20 09:00:32.585036 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 5/9] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:00:32 2022] GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b => generated 345 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:32.599793 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c33ce28b-c569-4c08-bc4b-c3b4f562f972 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Action: 'create', calling method: .version_select at 0x7f5afc890ca0>, body: {"snapshot": {"volume_id": "72658e36-60f3-4037-9c16-dc7454a4202b", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-961225892"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:00:33.073931 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c33ce28b-c569-4c08-bc4b-c3b4f562f972 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.478962 Oct 20 09:00:33.082849 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 5/10] 10.222.0.36 () {60 vars in 1162 bytes} [Thu Oct 20 09:00:32 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:33.093655 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4cea658d-3bc2-4c8e-bd2b-7c53b2586528 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] Calling method '.version_select at 0x7f5afc81ea60>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:33.101443 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f691da28-97fe-4249-9048-f654b88e7014 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc820c10>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:33.292123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-4cea658d-3bc2-4c8e-bd2b-7c53b2586528 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] HTTP exception thrown: Volume 3c3218ab-b726-4003-b1ed-bf128cd15960 could not be found. Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4cea658d-3bc2-4c8e-bd2b-7c53b2586528 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] Request method failure captured: Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960 HTTP/1.1 Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ4qRqIVC34YfRBAkaFO1gtr17PESxSW4vFYQamkydqQ9mFwj__-d-RgBRlBF1jIobxViGqH_COEDsaqQigeXUE5kny46O7z-HqyNHKpPGWVYGtKrzBau0-_0lSJba1BZW8isuYEP4fZVA6spxy3d5g3oKOFK9uG-q9O4eKQTRVk0pk Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: a393e550e71e4fa8a0ff9edd71382009 Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-VolumesGetTestJSON-669674540 Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:00:33.304129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/a393e550e71e4fa8a0ff9edd71382009"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_a393e550e71e4fa8a0ff9edd71382009", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/a393e550e71e4fa8a0ff9edd71382009"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/a393e550e71e4fa8a0ff9edd71382009"}]}] Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-VolumesGetTestJSON-669674540 Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: a393e550e71e4fa8a0ff9edd71382009 Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-VolumesGetTestJSON-669674540 Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-VolumesGetTestJSON-669674540-project Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 65f1034ab66f40dcb04da2872cfd64ef Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-VolumesGetTestJSON-669674540-project Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Volume 3c3218ab-b726-4003-b1ed-bf128cd15960 could not be found. Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '3c3218ab-b726-4003-b1ed-bf128cd15960'} Oct 20 09:00:33.305624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 3c3218ab-b726-4003-b1ed-bf128cd15960 could not be found. (HTTP 404) (Request-ID: req-8fd419d3-b255-48f2-90b6-7dd41d4ad45b) Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:33.307646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 20 09:00:33.309174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 3c3218ab-b726-4003-b1ed-bf128cd15960 could not be found. Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 20 09:00:33.311671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 3c3218ab-b726-4003-b1ed-bf128cd15960 could not be found. Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 3c3218ab-b726-4003-b1ed-bf128cd15960 could not be found. Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4cea658d-3bc2-4c8e-bd2b-7c53b2586528 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] Returning 404 to user: Volume 3c3218ab-b726-4003-b1ed-bf128cd15960 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:00:33.314534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4cea658d-3bc2-4c8e-bd2b-7c53b2586528 tempest-VolumesGetTestJSON-669674540 tempest-VolumesGetTestJSON-669674540-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960" status: 404 len: 109 microversion: 2.1 time: 0.229412 Oct 20 09:00:33.322423 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 6/11] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:00:33 2022] GET /compute/v2.1/os-volumes/3c3218ab-b726-4003-b1ed-bf128cd15960 => generated 109 bytes in 238 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:00:33.395521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f691da28-97fe-4249-9048-f654b88e7014 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6" status: 200 len: 286 microversion: 2.1 time: 0.309262 Oct 20 09:00:33.396265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1251 bytes} [Thu Oct 20 09:00:33 2022] GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6 => generated 286 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:33.412516 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7b66f7b3-fd6c-4f3d-9ee2-cabef431057d tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc80c1f0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:33.497803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7b66f7b3-fd6c-4f3d-9ee2-cabef431057d tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.093860 Oct 20 09:00:33.498598 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1140 bytes} [Thu Oct 20 09:00:33 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:33.510793 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4ff0dc11-d208-46e3-b88d-1d754eb4c807 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc82a3a0>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:33.572896 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4ff0dc11-d208-46e3-b88d-1d754eb4c807 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6" status: 200 len: 286 microversion: 2.1 time: 0.066308 Oct 20 09:00:33.573893 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 7/14] 10.222.0.36 () {58 vars in 1251 bytes} [Thu Oct 20 09:00:33 2022] GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6 => generated 286 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:34.595594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ffa20df3-262b-4dee-a5d4-1dec9f42bf4e tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc80f4c0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:34.656190 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ffa20df3-262b-4dee-a5d4-1dec9f42bf4e tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6" status: 200 len: 286 microversion: 2.1 time: 0.073714 Oct 20 09:00:34.657018 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 8/15] 10.222.0.36 () {58 vars in 1251 bytes} [Thu Oct 20 09:00:34 2022] GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6 => generated 286 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:35.670050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fa69897e-2a8e-4cf8-9830-b8471b1d19d0 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc820c10>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:35.791185 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fa69897e-2a8e-4cf8-9830-b8471b1d19d0 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6" status: 200 len: 286 microversion: 2.1 time: 0.125150 Oct 20 09:00:35.792059 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 8/16] 10.222.0.36 () {58 vars in 1251 bytes} [Thu Oct 20 09:00:35 2022] GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6 => generated 286 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:36.814390 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-adc9c2a9-f6e5-4861-9b00-c935f2a8d056 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc80ce50>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:36.879300 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-adc9c2a9-f6e5-4861-9b00-c935f2a8d056 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6" status: 200 len: 286 microversion: 2.1 time: 0.078225 Oct 20 09:00:36.887611 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 9/17] 10.222.0.36 () {58 vars in 1251 bytes} [Thu Oct 20 09:00:36 2022] GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6 => generated 286 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:37.896714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6c6bd748-1531-442b-a028-debf2894fbd9 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc820a60>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:38.004122 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6c6bd748-1531-442b-a028-debf2894fbd9 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6" status: 200 len: 287 microversion: 2.1 time: 0.114120 Oct 20 09:00:38.005097 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 9/18] 10.222.0.36 () {58 vars in 1251 bytes} [Thu Oct 20 09:00:37 2022] GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6 => generated 287 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:38.016581 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ddf8428a-bd2b-4d97-b65a-3043a4333361 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc80c790>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:38.186805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ddf8428a-bd2b-4d97-b65a-3043a4333361 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "DELETE /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6" status: 202 len: 0 microversion: 2.1 time: 0.174039 Oct 20 09:00:38.187637 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 10/19] 10.222.0.36 () {58 vars in 1254 bytes} [Thu Oct 20 09:00:38 2022] DELETE /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6 => generated 0 bytes in 175 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:00:38.197080 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-70fa11ed-8120-495c-b9f4-60ba8f8095c9 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc890af0>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:38.288513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-70fa11ed-8120-495c-b9f4-60ba8f8095c9 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6" status: 200 len: 286 microversion: 2.1 time: 0.096310 Oct 20 09:00:38.289080 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 10/20] 10.222.0.36 () {58 vars in 1251 bytes} [Thu Oct 20 09:00:38 2022] GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6 => generated 286 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:39.302495 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3e03e002-9434-445d-ab4c-14ff167890ad tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc80c040>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:39.353648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-3e03e002-9434-445d-ab4c-14ff167890ad tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] HTTP exception thrown: Snapshot 2aca7b2a-5fa1-450e-b215-31b810c1e6e6 could not be found. Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3e03e002-9434-445d-ab4c-14ff167890ad tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Request method failure captured: Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6 HTTP/1.1 Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ4q8zLubVp4CVbbRJIFWlJqB6eQXm15Y97CmBl_r6EFW3-xW9Dw-10tq6Oqqi96xAs6xS8bDfIlY5IIJrvKb9BqOyzbM0-UzVhivJ86MD4H15s4okZL8n4-guRbWmjMB3Gxx9seHaJ4m_pwRxuz1pqVexECRJbSLZWTonbuJ-dlUdA Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 16dda493652d4c87b075ee8cef7a8a95 Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:00:39.358350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/16dda493652d4c87b075ee8cef7a8a95"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_16dda493652d4c87b075ee8cef7a8a95", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/16dda493652d4c87b075ee8cef7a8a95"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/16dda493652d4c87b075ee8cef7a8a95"}]}] Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 16dda493652d4c87b075ee8cef7a8a95 Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-VolumesSnapshotsTestJSON-1905269572-project Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8fbb2f982ece43a289217e5a6c16f3ab Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1905269572-project Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Snapshot 2aca7b2a-5fa1-450e-b215-31b810c1e6e6 could not be found. Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '2aca7b2a-5fa1-450e-b215-31b810c1e6e6'} Oct 20 09:00:39.360081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 2aca7b2a-5fa1-450e-b215-31b810c1e6e6 could not be found. (HTTP 404) (Request-ID: req-58f9b4ba-a9aa-431b-ba24-f46073acad64) Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:39.361237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 20 09:00:39.363406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 2aca7b2a-5fa1-450e-b215-31b810c1e6e6 could not be found. Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 20 09:00:39.365419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 2aca7b2a-5fa1-450e-b215-31b810c1e6e6 could not be found. Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 2aca7b2a-5fa1-450e-b215-31b810c1e6e6 could not be found. Oct 20 09:00:39.368708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:39.373027 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3e03e002-9434-445d-ab4c-14ff167890ad tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Returning 404 to user: Snapshot 2aca7b2a-5fa1-450e-b215-31b810c1e6e6 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:00:39.373027 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3e03e002-9434-445d-ab4c-14ff167890ad tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6" status: 404 len: 111 microversion: 2.1 time: 0.073455 Oct 20 09:00:39.373027 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 11/21] 10.222.0.36 () {58 vars in 1251 bytes} [Thu Oct 20 09:00:39 2022] GET /compute/v2.1/os-snapshots/2aca7b2a-5fa1-450e-b215-31b810c1e6e6 => generated 111 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:00:39.392639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b01f3591-a207-413d-9b05-631b5b223fec tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc890f70>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:39.656521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b01f3591-a207-413d-9b05-631b5b223fec tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "DELETE /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b" status: 202 len: 0 microversion: 2.1 time: 0.279965 Oct 20 09:00:39.657705 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 11/22] 10.222.0.36 () {58 vars in 1248 bytes} [Thu Oct 20 09:00:39 2022] DELETE /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b => generated 0 bytes in 282 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:00:39.672137 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b7966999-4f74-49a7-8460-106374b70ea7 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc80ca60>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:39.857089 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b7966999-4f74-49a7-8460-106374b70ea7 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b" status: 200 len: 344 microversion: 2.1 time: 0.194820 Oct 20 09:00:39.857743 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 12/23] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:00:39 2022] GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b => generated 344 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:40.297689 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1919555677", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "networks": [{"uuid": "47c6132f-de00-464a-9c0d-6ac6d3d24b81"}]}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:00:40.634279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] validate_networks() for [('47c6132f-de00-464a-9c0d-6ac6d3d24b81', None, None, None, None, None)] {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:00:40.884942 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5c7da1aa-1b2c-4a45-8f8a-8a97de54f0c5 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc80f280>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:40.977153 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-5c7da1aa-1b2c-4a45-8f8a-8a97de54f0c5 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] HTTP exception thrown: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5c7da1aa-1b2c-4a45-8f8a-8a97de54f0c5 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Request method failure captured: Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b HTTP/1.1 Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ4q8zLubVp4CVbbRJIFWlJqB6eQXm15Y97CmBl_r6EFW3-xW9Dw-10tq6Oqqi96xAs6xS8bDfIlY5IIJrvKb9BqOyzbM0-UzVhivJ86MD4H15s4okZL8n4-guRbWmjMB3Gxx9seHaJ4m_pwRxuz1pqVexECRJbSLZWTonbuJ-dlUdA Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 16dda493652d4c87b075ee8cef7a8a95 Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:00:40.979594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/16dda493652d4c87b075ee8cef7a8a95"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_16dda493652d4c87b075ee8cef7a8a95", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/16dda493652d4c87b075ee8cef7a8a95"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/16dda493652d4c87b075ee8cef7a8a95"}]}] Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 16dda493652d4c87b075ee8cef7a8a95 Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-VolumesSnapshotsTestJSON-1905269572-project Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8fbb2f982ece43a289217e5a6c16f3ab Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1905269572-project Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '72658e36-60f3-4037-9c16-dc7454a4202b'} Oct 20 09:00:40.980976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. (HTTP 404) (Request-ID: req-1c3a470d-158f-4867-b54c-5aa6042264aa) Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:40.981805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 20 09:00:40.982989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 20 09:00:40.984103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5c7da1aa-1b2c-4a45-8f8a-8a97de54f0c5 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Returning 404 to user: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5c7da1aa-1b2c-4a45-8f8a-8a97de54f0c5 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b" status: 404 len: 109 microversion: 2.1 time: 0.112768 Oct 20 09:00:40.985319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 13/24] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:00:40 2022] GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b => generated 109 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:00:40.997496 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-06b993f0-f6a6-4892-a523-1342b8bca841 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc80c820>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:41.096815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-06b993f0-f6a6-4892-a523-1342b8bca841 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] HTTP exception thrown: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-06b993f0-f6a6-4892-a523-1342b8bca841 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Request method failure captured: Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: DELETE /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b HTTP/1.1 Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ4q8zLubVp4CVbbRJIFWlJqB6eQXm15Y97CmBl_r6EFW3-xW9Dw-10tq6Oqqi96xAs6xS8bDfIlY5IIJrvKb9BqOyzbM0-UzVhivJ86MD4H15s4okZL8n4-guRbWmjMB3Gxx9seHaJ4m_pwRxuz1pqVexECRJbSLZWTonbuJ-dlUdA Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 16dda493652d4c87b075ee8cef7a8a95 Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:00:41.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/16dda493652d4c87b075ee8cef7a8a95"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_16dda493652d4c87b075ee8cef7a8a95", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/16dda493652d4c87b075ee8cef7a8a95"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/16dda493652d4c87b075ee8cef7a8a95"}]}] Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 16dda493652d4c87b075ee8cef7a8a95 Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-VolumesSnapshotsTestJSON-1905269572-project Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8fbb2f982ece43a289217e5a6c16f3ab Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1905269572-project Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '72658e36-60f3-4037-9c16-dc7454a4202b'} Oct 20 09:00:41.100838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. (HTTP 404) (Request-ID: req-8b691431-1522-40f2-ace8-28e16d9ba8bd) Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:41.104972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 20 09:00:41.106223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 20 09:00:41.107426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-06b993f0-f6a6-4892-a523-1342b8bca841 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Returning 404 to user: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-06b993f0-f6a6-4892-a523-1342b8bca841 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "DELETE /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b" status: 404 len: 109 microversion: 2.1 time: 0.109615 Oct 20 09:00:41.108566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 14/25] 10.222.0.36 () {58 vars in 1248 bytes} [Thu Oct 20 09:00:40 2022] DELETE /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b => generated 109 bytes in 111 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 20 09:00:41.276552 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1900711614", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:00:41.549093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] validate_networks() for [('89681d33-1e95-46d3-a4f1-4813c7b75e62', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:00:43.551101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:00:43.551674 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:00:43.551992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:00:43.552402 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:00:43.553796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:00:43.574647 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:43.575445 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:43.576799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:43.578114 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:43.579315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:43.580606 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:43.635092 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Getting quotas for project d24a1fc0e9344af181d49cfc58f0c88b. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:00:43.643994 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Getting quotas for user 7f932695a5994c5790d14c210ea65326 and project d24a1fc0e9344af181d49cfc58f0c88b. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:00:43.663600 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:00:43.667407 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:00:43.721342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:00:43.721852 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:00:43.722218 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:00:43.722973 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:00:43.723408 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:00:43.788984 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:43.789476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:43.789950 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:43.790701 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:43.791149 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:43.791567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:43.819782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-cbf40ca3-254f-4713-b6cb-0a2a6933abda tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.716366 Oct 20 09:00:43.820452 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 15/26] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:00:41 2022] POST /compute/v2.1/servers => generated 383 bytes in 2719 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:00:43.835435 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8a29597d-1486-49a3-aac0-2683c82d9e32 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Calling method '.version_select at 0x7f5afc70d9d0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:43.909715 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Getting quotas for project 4da29c9977ca44baa867b3c7fc86164b. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:00:43.911248 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-8a29597d-1486-49a3-aac0-2683c82d9e32 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] HTTP exception thrown: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8a29597d-1486-49a3-aac0-2683c82d9e32 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Request method failure captured: Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b HTTP/1.1 Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ4q8zLubVp4CVbbRJIFWlJqB6eQXm15Y97CmBl_r6EFW3-xW9Dw-10tq6Oqqi96xAs6xS8bDfIlY5IIJrvKb9BqOyzbM0-UzVhivJ86MD4H15s4okZL8n4-guRbWmjMB3Gxx9seHaJ4m_pwRxuz1pqVexECRJbSLZWTonbuJ-dlUdA Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 16dda493652d4c87b075ee8cef7a8a95 Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:00:43.914441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/16dda493652d4c87b075ee8cef7a8a95"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_16dda493652d4c87b075ee8cef7a8a95", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/16dda493652d4c87b075ee8cef7a8a95"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/16dda493652d4c87b075ee8cef7a8a95"}]}] Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 16dda493652d4c87b075ee8cef7a8a95 Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1905269572 Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-VolumesSnapshotsTestJSON-1905269572-project Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8fbb2f982ece43a289217e5a6c16f3ab Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1905269572-project Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '72658e36-60f3-4037-9c16-dc7454a4202b'} Oct 20 09:00:43.915836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. (HTTP 404) (Request-ID: req-4e994081-5564-41c0-aebb-2b40d16ad4f0) Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:43.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 20 09:00:43.917915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 20 09:00:43.919064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8a29597d-1486-49a3-aac0-2683c82d9e32 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] Returning 404 to user: Volume 72658e36-60f3-4037-9c16-dc7454a4202b could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8a29597d-1486-49a3-aac0-2683c82d9e32 tempest-VolumesSnapshotsTestJSON-1905269572 tempest-VolumesSnapshotsTestJSON-1905269572-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b" status: 404 len: 109 microversion: 2.1 time: 0.093798 Oct 20 09:00:43.920739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 16/27] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:00:43 2022] GET /compute/v2.1/os-volumes/72658e36-60f3-4037-9c16-dc7454a4202b => generated 109 bytes in 95 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:00:43.945450 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Getting quotas for user c7c40a4592874732871eaabd04d8662c and project 4da29c9977ca44baa867b3c7fc86164b. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:00:44.008986 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:00:44.018761 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:00:44.211713 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b8da6656-1354-4765-b1ae-f16741f8f8aa tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.257746 Oct 20 09:00:44.227980 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 12/28] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:00:39 2022] POST /compute/v2.1/servers => generated 380 bytes in 4275 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:00:44.243636 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-75e430bb-f60e-4e44-b927-fcd02de0c0a6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:44.248859 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-75e430bb-f60e-4e44-b927-fcd02de0c0a6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:44.512868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-75e430bb-f60e-4e44-b927-fcd02de0c0a6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:44.539851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-75e430bb-f60e-4e44-b927-fcd02de0c0a6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1222 microversion: 2.1 time: 0.309548 Oct 20 09:00:44.540961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 13/29] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:44 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1222 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:44.559616 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-30590110-e09a-48b0-a48d-c68858565802 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:44.561417 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5b5998c0-20e6-4172-8a41-b65fe60c18b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Action: 'create', calling method: .version_select at 0x7f5afc70d430>, body: {"keypair": {"name": "tempest-keypair-121686616", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC44eD6RFa5GtuT1vLJUBqW4kmuBRYW0MOWj6Oduj27vbxLZ5gd4gA8xdsdfgmyZ9Rl2IUO7V4i+TRNcAGfD7AapCsOJNChZ3AI5Vft22EbghEV9AnUq+8JmlMq6bmShJg=="}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:00:44.565227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-30590110-e09a-48b0-a48d-c68858565802 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:44.615834 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-5b5998c0-20e6-4172-8a41-b65fe60c18b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Getting quotas for project 869b5cf3a9264fd3b70ae656d7814cfb. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:00:44.625794 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-5b5998c0-20e6-4172-8a41-b65fe60c18b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Getting quotas for user fd9733d1bb2b4520b70090542615ddf9 and project 869b5cf3a9264fd3b70ae656d7814cfb. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:00:44.698587 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-30590110-e09a-48b0-a48d-c68858565802 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:44.706782 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-30590110-e09a-48b0-a48d-c68858565802 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1215 microversion: 2.1 time: 0.161213 Oct 20 09:00:44.707697 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 14/30] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:44 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1215 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:44.738862 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5b5998c0-20e6-4172-8a41-b65fe60c18b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.820517 Oct 20 09:00:44.741322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 17/31] 10.222.0.36 () {62 vars in 1200 bytes} [Thu Oct 20 09:00:43 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 824 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 20 09:00:45.331174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-855127199", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:00:45.492104 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] validate_networks() for [('89681d33-1e95-46d3-a4f1-4813c7b75e62', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:00:45.754766 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c9d92487-55d0-4691-8aee-0a06cd3b4ea8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: .version_select at 0x7f5afbee6d30>, body: {"keypair": {"name": "tempest-keypair-961266284", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNQu9Z0TARLExuzZXWWjC0RZVwHcCxqXgm5Od3SryfIUcaZ1UDbQAabrytuEQGB349jy73hrHHJ5Yf8nqmXNUPQKI+APFZQIqGuC3H8B/uQe7VF6mearkPL4iY9DOz7ehQ=="}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:00:45.855055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-c9d92487-55d0-4691-8aee-0a06cd3b4ea8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'key_pairs'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:00:45.874836 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-c9d92487-55d0-4691-8aee-0a06cd3b4ea8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for user 8002ab26f6d64822aaa65aa730781b76 and project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'key_pairs'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:00:45.969334 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c9d92487-55d0-4691-8aee-0a06cd3b4ea8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 1.140181 Oct 20 09:00:45.970983 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 15/32] 10.222.0.36 () {60 vars in 1159 bytes} [Thu Oct 20 09:00:44 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 1143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:00:45.994643 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-66d949ed-50a3-4f62-9831-947927bf33c9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:46.000333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-66d949ed-50a3-4f62-9831-947927bf33c9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:46.750646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-66d949ed-50a3-4f62-9831-947927bf33c9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:46.760366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-66d949ed-50a3-4f62-9831-947927bf33c9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1215 microversion: 2.1 time: 0.777785 Oct 20 09:00:46.764420 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 16/33] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:45 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1215 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:47.710934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:00:47.711449 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:00:47.712657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:00:47.713233 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:00:47.713752 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:00:47.738285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:47.738837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:47.740073 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:47.741554 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:47.742355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:47.743499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:47.794248 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f6b95095-7faa-4732-9586-c20f5c02ac68 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:47.797848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f6b95095-7faa-4732-9586-c20f5c02ac68 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:47.808148 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Getting quotas for project d24a1fc0e9344af181d49cfc58f0c88b. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:00:47.813640 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f6b95095-7faa-4732-9586-c20f5c02ac68 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:47.814750 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f6b95095-7faa-4732-9586-c20f5c02ac68 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:47.817624 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f6b95095-7faa-4732-9586-c20f5c02ac68 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:47.820144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f6b95095-7faa-4732-9586-c20f5c02ac68 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:47.820821 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Getting quotas for user 7f932695a5994c5790d14c210ea65326 and project d24a1fc0e9344af181d49cfc58f0c88b. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:00:47.823112 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f6b95095-7faa-4732-9586-c20f5c02ac68 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:47.823655 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f6b95095-7faa-4732-9586-c20f5c02ac68 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:47.849786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:00:47.861841 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:00:47.990123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c609fb57-4786-49ee-8461-c1e33405662f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.665245 Oct 20 09:00:47.993026 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 18/34] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:00:45 2022] POST /compute/v2.1/servers => generated 383 bytes in 2669 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 20 09:00:48.006687 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8ee264db-e8a8-42f2-9967-4d6d34726c45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:48.011307 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8ee264db-e8a8-42f2-9967-4d6d34726c45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:48.398053 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8ee264db-e8a8-42f2-9967-4d6d34726c45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:48.419387 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8ee264db-e8a8-42f2-9967-4d6d34726c45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1221 microversion: 2.1 time: 0.418050 Oct 20 09:00:48.420256 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 19/35] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:48 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1221 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:48.623907 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f6b95095-7faa-4732-9586-c20f5c02ac68 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:48.656197 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f6b95095-7faa-4732-9586-c20f5c02ac68 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1211 microversion: 2.1 time: 0.868351 Oct 20 09:00:48.656786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 17/36] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:47 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1211 bytes in 870 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:49.449420 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-00ad4be6-3aec-4af5-a066-2479e568f7b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:49.452336 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-00ad4be6-3aec-4af5-a066-2479e568f7b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:49.476473 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-00ad4be6-3aec-4af5-a066-2479e568f7b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:49.478693 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-00ad4be6-3aec-4af5-a066-2479e568f7b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:49.480719 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-00ad4be6-3aec-4af5-a066-2479e568f7b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:49.486948 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-00ad4be6-3aec-4af5-a066-2479e568f7b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:49.489955 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-00ad4be6-3aec-4af5-a066-2479e568f7b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:49.490628 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-00ad4be6-3aec-4af5-a066-2479e568f7b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:49.563480 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-257863942", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8e03c888-668e-46f5-b105-1902608fb69d"}], "security_groups": [{"name": "tempest-securitygroup--114675733"}], "key_name": "tempest-keypair-121686616", "user_data": "}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:00:49.922999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] validate_networks() for [('8e03c888-668e-46f5-b105-1902608fb69d', None, None, None, None, None)] {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:00:50.095817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-00ad4be6-3aec-4af5-a066-2479e568f7b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:50.132798 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-00ad4be6-3aec-4af5-a066-2479e568f7b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1217 microversion: 2.1 time: 0.690073 Oct 20 09:00:50.138613 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 20/37] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:49 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1217 bytes in 698 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:50.143666 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d312fa35-724d-4a9f-a4cc-028702310cb7 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:50.160993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d312fa35-724d-4a9f-a4cc-028702310cb7 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:50.186663 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d312fa35-724d-4a9f-a4cc-028702310cb7 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:50.187176 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d312fa35-724d-4a9f-a4cc-028702310cb7 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:50.187622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d312fa35-724d-4a9f-a4cc-028702310cb7 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:50.210548 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d312fa35-724d-4a9f-a4cc-028702310cb7 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:50.215337 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d312fa35-724d-4a9f-a4cc-028702310cb7 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:50.216125 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d312fa35-724d-4a9f-a4cc-028702310cb7 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:50.630017 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d312fa35-724d-4a9f-a4cc-028702310cb7 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:50.651391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d312fa35-724d-4a9f-a4cc-028702310cb7 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1267 microversion: 2.1 time: 0.508762 Oct 20 09:00:50.652008 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 21/38] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:50 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1267 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:51.160872 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c3fe004c-777b-436d-9ec5-4af32fdabdd3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:51.161568 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c3fe004c-777b-436d-9ec5-4af32fdabdd3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:51.176993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c3fe004c-777b-436d-9ec5-4af32fdabdd3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:51.184927 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c3fe004c-777b-436d-9ec5-4af32fdabdd3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:51.186233 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c3fe004c-777b-436d-9ec5-4af32fdabdd3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:51.187110 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c3fe004c-777b-436d-9ec5-4af32fdabdd3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:51.187588 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c3fe004c-777b-436d-9ec5-4af32fdabdd3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:51.188115 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c3fe004c-777b-436d-9ec5-4af32fdabdd3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:51.464304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c3fe004c-777b-436d-9ec5-4af32fdabdd3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:51.485807 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c3fe004c-777b-436d-9ec5-4af32fdabdd3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1273 microversion: 2.1 time: 0.328703 Oct 20 09:00:51.502266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 22/39] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:51 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1273 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:00:51.682680 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9f153c42-b725-4684-971c-fffb72ea99e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:51.686811 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9f153c42-b725-4684-971c-fffb72ea99e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:51.707911 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f153c42-b725-4684-971c-fffb72ea99e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:51.709134 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f153c42-b725-4684-971c-fffb72ea99e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:51.711328 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f153c42-b725-4684-971c-fffb72ea99e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:51.712328 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f153c42-b725-4684-971c-fffb72ea99e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:51.715080 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f153c42-b725-4684-971c-fffb72ea99e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:51.717468 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f153c42-b725-4684-971c-fffb72ea99e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:51.886011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9f153c42-b725-4684-971c-fffb72ea99e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:51.901113 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9f153c42-b725-4684-971c-fffb72ea99e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1273 microversion: 2.1 time: 0.227722 Oct 20 09:00:51.905196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 23/40] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:51 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1273 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:52.059307 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-197660637", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b8327be3-f028-4ebc-9b25-81152867d29e"}], "security_groups": [{"name": "tempest-securitygroup--800318894"}], "key_name": "tempest-keypair-961266284", "user_data": "}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:00:52.504317 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] validate_networks() for [('b8327be3-f028-4ebc-9b25-81152867d29e', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:00:53.863238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:00:54.196969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:00:54.197723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:00:54.197858 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:00:54.198248 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:00:54.198764 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:00:54.227559 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:54.237845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:54.239556 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:54.243871 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:54.245813 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:54.247349 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:54.344443 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:00:54.374303 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for user 8002ab26f6d64822aaa65aa730781b76 and project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:00:54.450993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:00:54.453482 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:00:54.585866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c307182e-0e38-4452-8b60-c4dc5bedb7cd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 2.531278 Oct 20 09:00:54.593654 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 24/41] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:00:52 2022] POST /compute/v2.1/servers => generated 401 bytes in 2540 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:00:54.599856 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-90615712-8a9f-4430-bb0f-f17ebdf42500 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:54.600421 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-90615712-8a9f-4430-bb0f-f17ebdf42500 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:54.609467 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-90615712-8a9f-4430-bb0f-f17ebdf42500 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:54.610596 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-90615712-8a9f-4430-bb0f-f17ebdf42500 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:54.611270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-90615712-8a9f-4430-bb0f-f17ebdf42500 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:54.612206 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-90615712-8a9f-4430-bb0f-f17ebdf42500 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:54.612795 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-90615712-8a9f-4430-bb0f-f17ebdf42500 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:54.613596 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-90615712-8a9f-4430-bb0f-f17ebdf42500 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:55.077391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-90615712-8a9f-4430-bb0f-f17ebdf42500 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:55.103932 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-90615712-8a9f-4430-bb0f-f17ebdf42500 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1279 microversion: 2.1 time: 0.508492 Oct 20 09:00:55.104829 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 25/42] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:54 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1279 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:55.109759 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8351da27-2cff-4579-8491-316b77ec7379 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:55.110701 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8351da27-2cff-4579-8491-316b77ec7379 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:55.126054 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8351da27-2cff-4579-8491-316b77ec7379 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:55.126918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8351da27-2cff-4579-8491-316b77ec7379 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:55.127619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8351da27-2cff-4579-8491-316b77ec7379 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:55.128547 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8351da27-2cff-4579-8491-316b77ec7379 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:55.135896 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8351da27-2cff-4579-8491-316b77ec7379 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:55.136873 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8351da27-2cff-4579-8491-316b77ec7379 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:55.298234 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:00:55.299498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:00:55.300238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:00:55.300907 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:00:55.301634 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:00:55.329450 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:55.330431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:55.331152 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:55.333254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:55.339409 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:55.341555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:55.424475 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8351da27-2cff-4579-8491-316b77ec7379 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:55.442391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8351da27-2cff-4579-8491-316b77ec7379 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1273 microversion: 2.1 time: 0.336233 Oct 20 09:00:55.445420 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 26/43] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:55 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1273 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:55.451428 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Getting quotas for project 869b5cf3a9264fd3b70ae656d7814cfb. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:00:55.472177 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Getting quotas for user fd9733d1bb2b4520b70090542615ddf9 and project 869b5cf3a9264fd3b70ae656d7814cfb. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:00:55.473895 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6782d348-5968-4cc4-b331-be763d6e30b4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:55.481277 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6782d348-5968-4cc4-b331-be763d6e30b4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:55.533828 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:00:55.548210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:00:55.702275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5614d778-8e8a-4f93-ac74-e90d9933a9a4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 6.143878 Oct 20 09:00:55.713036 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 18/44] 10.222.0.36 () {62 vars in 1188 bytes} [Thu Oct 20 09:00:49 2022] POST /compute/v2.1/servers => generated 401 bytes in 6155 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Oct 20 09:00:55.718221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-65308954-8856-43d0-960f-487eae4b41e2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:55.723074 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-65308954-8856-43d0-960f-487eae4b41e2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:55.869093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6782d348-5968-4cc4-b331-be763d6e30b4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:55.880571 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6782d348-5968-4cc4-b331-be763d6e30b4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1240 microversion: 2.1 time: 0.430968 Oct 20 09:00:55.881437 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 27/45] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:55 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1240 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:56.135780 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-455d988e-b927-4046-b86d-128a382f4caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:56.137358 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-455d988e-b927-4046-b86d-128a382f4caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:56.147722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-455d988e-b927-4046-b86d-128a382f4caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:56.150260 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-455d988e-b927-4046-b86d-128a382f4caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:56.150775 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-455d988e-b927-4046-b86d-128a382f4caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:56.151550 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-455d988e-b927-4046-b86d-128a382f4caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:56.152015 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-455d988e-b927-4046-b86d-128a382f4caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:56.152486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-455d988e-b927-4046-b86d-128a382f4caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:56.358373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65308954-8856-43d0-960f-487eae4b41e2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:56.368926 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65308954-8856-43d0-960f-487eae4b41e2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:56.372243 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65308954-8856-43d0-960f-487eae4b41e2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:56.375775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-65308954-8856-43d0-960f-487eae4b41e2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1280 microversion: 2.20 time: 0.661697 Oct 20 09:00:56.381758 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 19/46] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:00:55 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1280 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:00:56.474265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-44b9ad95-71ca-40ad-b8e7-37f09547b410 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:56.475216 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-44b9ad95-71ca-40ad-b8e7-37f09547b410 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:56.491578 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-44b9ad95-71ca-40ad-b8e7-37f09547b410 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:56.493494 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-44b9ad95-71ca-40ad-b8e7-37f09547b410 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:56.495277 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-44b9ad95-71ca-40ad-b8e7-37f09547b410 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:56.498550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-44b9ad95-71ca-40ad-b8e7-37f09547b410 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:56.499159 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-44b9ad95-71ca-40ad-b8e7-37f09547b410 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:56.499684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-44b9ad95-71ca-40ad-b8e7-37f09547b410 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:56.846228 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-455d988e-b927-4046-b86d-128a382f4caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:56.860417 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-455d988e-b927-4046-b86d-128a382f4caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.732600 Oct 20 09:00:56.866089 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 28/47] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:56 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:00:56.913559 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5aa9a718-786e-403c-9fb6-8a1880168aa9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:56.918698 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5aa9a718-786e-403c-9fb6-8a1880168aa9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:56.922116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa9a718-786e-403c-9fb6-8a1880168aa9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:56.933282 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa9a718-786e-403c-9fb6-8a1880168aa9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:56.936027 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa9a718-786e-403c-9fb6-8a1880168aa9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:56.936897 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa9a718-786e-403c-9fb6-8a1880168aa9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:56.937578 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa9a718-786e-403c-9fb6-8a1880168aa9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:56.938755 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa9a718-786e-403c-9fb6-8a1880168aa9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:57.656301 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5aa9a718-786e-403c-9fb6-8a1880168aa9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:57.687569 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5aa9a718-786e-403c-9fb6-8a1880168aa9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1292 microversion: 2.1 time: 0.777883 Oct 20 09:00:57.693929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 29/48] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:56 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1292 bytes in 785 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:57.720732 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:57.721720 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:57.741474 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:57.742303 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:57.742995 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:57.744824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:57.747075 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:57.747820 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:57.971230 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:57.987355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:57.989505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:57.992350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bb926f9f-b6bd-43ed-99ca-e3827fe590da tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1284 microversion: 2.20 time: 0.285953 Oct 20 09:00:57.998250 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 30/49] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:00:57 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1284 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:00:58.010558 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1b2ed8d9-26ee-46b7-ab11-5fe525e01e4b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:58.011718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1b2ed8d9-26ee-46b7-ab11-5fe525e01e4b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:58.024094 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b2ed8d9-26ee-46b7-ab11-5fe525e01e4b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:58.025203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b2ed8d9-26ee-46b7-ab11-5fe525e01e4b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:58.028229 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b2ed8d9-26ee-46b7-ab11-5fe525e01e4b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:58.029496 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b2ed8d9-26ee-46b7-ab11-5fe525e01e4b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:58.031053 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b2ed8d9-26ee-46b7-ab11-5fe525e01e4b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:58.032385 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b2ed8d9-26ee-46b7-ab11-5fe525e01e4b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:58.723194 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1b2ed8d9-26ee-46b7-ab11-5fe525e01e4b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:58.753155 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1b2ed8d9-26ee-46b7-ab11-5fe525e01e4b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.753469 Oct 20 09:00:58.764312 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 31/50] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:57 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 765 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:58.796464 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5ec2b180-b2c1-4c88-9e8e-6f343cd99a48 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:58.797267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5ec2b180-b2c1-4c88-9e8e-6f343cd99a48 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:58.829593 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ec2b180-b2c1-4c88-9e8e-6f343cd99a48 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:58.831616 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ec2b180-b2c1-4c88-9e8e-6f343cd99a48 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:58.832926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ec2b180-b2c1-4c88-9e8e-6f343cd99a48 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:58.837521 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ec2b180-b2c1-4c88-9e8e-6f343cd99a48 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:58.838697 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ec2b180-b2c1-4c88-9e8e-6f343cd99a48 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:58.840972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ec2b180-b2c1-4c88-9e8e-6f343cd99a48 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:59.051651 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5ec2b180-b2c1-4c88-9e8e-6f343cd99a48 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:59.073492 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5ec2b180-b2c1-4c88-9e8e-6f343cd99a48 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1298 microversion: 2.1 time: 0.306085 Oct 20 09:00:59.078209 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 32/51] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:58 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1298 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:00:59.111759 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:59.114005 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:59.149967 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:59.150945 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:59.167841 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:59.172020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:59.176219 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:59.177415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:59.469167 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-44b9ad95-71ca-40ad-b8e7-37f09547b410 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:59.498265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-44b9ad95-71ca-40ad-b8e7-37f09547b410 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 3.024801 Oct 20 09:00:59.500754 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 20/52] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:56 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 3031 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:00:59.666289 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:59.689474 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:59.692055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:00:59.695928 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6f8f7e82-d60b-476f-bb9d-5e8b1b8eefb0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1332 microversion: 2.20 time: 0.616468 Oct 20 09:00:59.710996 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 33/53] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:00:59 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1332 bytes in 626 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:00:59.802385 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-77574770-4768-4b44-ab07-38a55e7420c8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:00:59.812542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-77574770-4768-4b44-ab07-38a55e7420c8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:00:59.827425 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77574770-4768-4b44-ab07-38a55e7420c8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:59.828629 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77574770-4768-4b44-ab07-38a55e7420c8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:59.829452 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77574770-4768-4b44-ab07-38a55e7420c8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:00:59.832752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77574770-4768-4b44-ab07-38a55e7420c8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:00:59.833797 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77574770-4768-4b44-ab07-38a55e7420c8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:00:59.835825 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77574770-4768-4b44-ab07-38a55e7420c8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:00.107971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4cacad80-fd63-4420-b35b-f802695e6cbb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:00.108617 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4cacad80-fd63-4420-b35b-f802695e6cbb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:00.122036 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4cacad80-fd63-4420-b35b-f802695e6cbb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:00.124917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4cacad80-fd63-4420-b35b-f802695e6cbb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:00.125700 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4cacad80-fd63-4420-b35b-f802695e6cbb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:00.127085 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4cacad80-fd63-4420-b35b-f802695e6cbb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:00.129223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4cacad80-fd63-4420-b35b-f802695e6cbb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:00.132371 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4cacad80-fd63-4420-b35b-f802695e6cbb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:00.588303 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4cacad80-fd63-4420-b35b-f802695e6cbb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:00.618935 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4cacad80-fd63-4420-b35b-f802695e6cbb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1298 microversion: 2.1 time: 0.515702 Oct 20 09:01:00.620211 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 34/54] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:00 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1298 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:00.633660 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-53a2b6f5-29d7-4da4-967e-fc2b8265085b tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:00.648220 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-53a2b6f5-29d7-4da4-967e-fc2b8265085b tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:00.667899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53a2b6f5-29d7-4da4-967e-fc2b8265085b tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:00.673762 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53a2b6f5-29d7-4da4-967e-fc2b8265085b tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:00.675101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53a2b6f5-29d7-4da4-967e-fc2b8265085b tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:00.678102 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53a2b6f5-29d7-4da4-967e-fc2b8265085b tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:00.678855 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53a2b6f5-29d7-4da4-967e-fc2b8265085b tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:00.683364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53a2b6f5-29d7-4da4-967e-fc2b8265085b tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:01.134113 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-53a2b6f5-29d7-4da4-967e-fc2b8265085b tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:01.150792 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-53a2b6f5-29d7-4da4-967e-fc2b8265085b tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.527549 Oct 20 09:01:01.167110 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 35/55] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:00 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:01.189191 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:01.194243 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:01.216777 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:01.219206 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:01.219941 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:01.220605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:01.221519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:01.222252 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:01.222932 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:01.527706 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:01.546757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:01.551075 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:01.552589 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d4c40644-51d4-47aa-84a0-f8f1fdbcf471 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1338 microversion: 2.20 time: 0.383668 Oct 20 09:01:01.553900 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 36/56] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:01 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1338 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:01.647069 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-12dc9af9-a897-4bba-bb2b-fe39024daf8b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:01.648377 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-12dc9af9-a897-4bba-bb2b-fe39024daf8b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:01.666866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-12dc9af9-a897-4bba-bb2b-fe39024daf8b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:01.667714 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-12dc9af9-a897-4bba-bb2b-fe39024daf8b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:01.668369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-12dc9af9-a897-4bba-bb2b-fe39024daf8b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:01.669316 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-12dc9af9-a897-4bba-bb2b-fe39024daf8b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:01.670286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-12dc9af9-a897-4bba-bb2b-fe39024daf8b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:01.676525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-12dc9af9-a897-4bba-bb2b-fe39024daf8b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:01.740286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=100 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:01.920788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-12dc9af9-a897-4bba-bb2b-fe39024daf8b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:01.935948 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-12dc9af9-a897-4bba-bb2b-fe39024daf8b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1298 microversion: 2.1 time: 0.294354 Oct 20 09:01:01.937381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 37/57] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:01 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1298 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:02.196569 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9ca96b2c-89bd-4ed3-b011-63f2f1639c80 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:02.197881 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9ca96b2c-89bd-4ed3-b011-63f2f1639c80 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:02.219805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca96b2c-89bd-4ed3-b011-63f2f1639c80 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:02.220295 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca96b2c-89bd-4ed3-b011-63f2f1639c80 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:02.220731 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca96b2c-89bd-4ed3-b011-63f2f1639c80 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:02.221486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca96b2c-89bd-4ed3-b011-63f2f1639c80 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:02.222176 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca96b2c-89bd-4ed3-b011-63f2f1639c80 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:02.222792 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca96b2c-89bd-4ed3-b011-63f2f1639c80 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:02.341943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-77574770-4768-4b44-ab07-38a55e7420c8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:02.352644 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-77574770-4768-4b44-ab07-38a55e7420c8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 2.557834 Oct 20 09:01:02.373017 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 21/58] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:00:59 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 2579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:02.582762 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:02.583669 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:02.594729 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:02.596662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:02.597294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:02.606130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:02.617215 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:02.620655 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:03.180901 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9ca96b2c-89bd-4ed3-b011-63f2f1639c80 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:03.192231 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9ca96b2c-89bd-4ed3-b011-63f2f1639c80 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 1.006220 Oct 20 09:01:03.193639 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 38/59] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:02 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 1008 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:03.198583 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a5a36ecb-80e7-478c-9ed6-6f9664163e1e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:03.199935 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a5a36ecb-80e7-478c-9ed6-6f9664163e1e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:03.207172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a36ecb-80e7-478c-9ed6-6f9664163e1e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:03.207997 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a36ecb-80e7-478c-9ed6-6f9664163e1e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:03.208730 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a36ecb-80e7-478c-9ed6-6f9664163e1e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:03.209738 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a36ecb-80e7-478c-9ed6-6f9664163e1e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:03.210888 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a36ecb-80e7-478c-9ed6-6f9664163e1e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:03.211962 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a36ecb-80e7-478c-9ed6-6f9664163e1e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:03.345531 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:03.386865 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:03.388084 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:03.398392 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5730198a-474a-4b92-9721-3fca51e5b4aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1338 microversion: 2.20 time: 0.814546 Oct 20 09:01:03.416646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 22/60] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:02 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1338 bytes in 838 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:03.429572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ece63142-bf39-4040-9801-5f6c8d6a549b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:03.430232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ece63142-bf39-4040-9801-5f6c8d6a549b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:03.444016 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ece63142-bf39-4040-9801-5f6c8d6a549b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:03.447876 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ece63142-bf39-4040-9801-5f6c8d6a549b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:03.448356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ece63142-bf39-4040-9801-5f6c8d6a549b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:03.450617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ece63142-bf39-4040-9801-5f6c8d6a549b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:03.451105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ece63142-bf39-4040-9801-5f6c8d6a549b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:03.451544 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ece63142-bf39-4040-9801-5f6c8d6a549b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:03.841351 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ece63142-bf39-4040-9801-5f6c8d6a549b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:03.870680 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ece63142-bf39-4040-9801-5f6c8d6a549b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.449208 Oct 20 09:01:03.877089 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 23/61] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:03 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:04.257433 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a5a36ecb-80e7-478c-9ed6-6f9664163e1e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:04.280425 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a5a36ecb-80e7-478c-9ed6-6f9664163e1e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 1.085510 Oct 20 09:01:04.294186 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 39/62] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:03 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 1100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:04.300699 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=6 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:04.309249 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-95e8e515-f229-48ee-b3c9-a030c9c5db24 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:04.309866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-95e8e515-f229-48ee-b3c9-a030c9c5db24 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:04.345022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-95e8e515-f229-48ee-b3c9-a030c9c5db24 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:04.345541 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-95e8e515-f229-48ee-b3c9-a030c9c5db24 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:04.346053 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-95e8e515-f229-48ee-b3c9-a030c9c5db24 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:04.349105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-95e8e515-f229-48ee-b3c9-a030c9c5db24 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:04.349604 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-95e8e515-f229-48ee-b3c9-a030c9c5db24 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:04.350041 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-95e8e515-f229-48ee-b3c9-a030c9c5db24 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:04.593783 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-a72187ca-28c1-412f-9898-d229f58a785d req-5ae93bb7-25a7-4438-ae7c-10194b67f356 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6cabc1ab-03aa-4ff2-be06-d99f9efc6150", "tag": "761aed46-611d-4ca6-9fc5-7334f3e86a8f"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:04.626531 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-a72187ca-28c1-412f-9898-d229f58a785d req-5ae93bb7-25a7-4438-ae7c-10194b67f356 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:04.628564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-a72187ca-28c1-412f-9898-d229f58a785d req-5ae93bb7-25a7-4438-ae7c-10194b67f356 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:04.629239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-a72187ca-28c1-412f-9898-d229f58a785d req-5ae93bb7-25a7-4438-ae7c-10194b67f356 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:04.682699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-a72187ca-28c1-412f-9898-d229f58a785d req-5ae93bb7-25a7-4438-ae7c-10194b67f356 service nova] Creating event network-changed:761aed46-611d-4ca6-9fc5-7334f3e86a8f for instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 on hv-cinder-maste Oct 20 09:01:04.684569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-a72187ca-28c1-412f-9898-d229f58a785d req-5ae93bb7-25a7-4438-ae7c-10194b67f356 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=119068) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 20 09:01:04.686077 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-a72187ca-28c1-412f-9898-d229f58a785d req-5ae93bb7-25a7-4438-ae7c-10194b67f356 service nova] Acquired lock "compute-rpcapi-router" {{(pid=119068) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 20 09:01:04.689680 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-631606c6-6660-4740-acf8-91012ea7ac50 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:04.691895 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-631606c6-6660-4740-acf8-91012ea7ac50 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:04.697832 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-631606c6-6660-4740-acf8-91012ea7ac50 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:04.700884 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-631606c6-6660-4740-acf8-91012ea7ac50 None None] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:04.701624 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-631606c6-6660-4740-acf8-91012ea7ac50 None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:04.703161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-631606c6-6660-4740-acf8-91012ea7ac50 None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:04.734419 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.compute.rpcapi [None req-631606c6-6660-4740-acf8-91012ea7ac50 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 20 09:01:04.736072 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-631606c6-6660-4740-acf8-91012ea7ac50 None None] Releasing lock "compute-rpcapi-router" {{(pid=119068) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 20 09:01:04.737339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-95e8e515-f229-48ee-b3c9-a030c9c5db24 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:04.749990 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-95e8e515-f229-48ee-b3c9-a030c9c5db24 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.454000 Oct 20 09:01:04.751510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 40/63] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:04 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:04.767916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:04.771390 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:04.784214 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-631606c6-6660-4740-acf8-91012ea7ac50 None None] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.903917 Oct 20 09:01:04.789934 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 24/64] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 910 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:04.797766 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:04.798546 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:04.799455 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:04.804529 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:04.805255 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:04.805910 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:04.904373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-779a6d17-4ac3-4e20-abde-80fc7809dc40 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:04.905079 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-779a6d17-4ac3-4e20-abde-80fc7809dc40 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:04.922900 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-779a6d17-4ac3-4e20-abde-80fc7809dc40 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:04.924444 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-779a6d17-4ac3-4e20-abde-80fc7809dc40 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:04.925142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-779a6d17-4ac3-4e20-abde-80fc7809dc40 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:04.926413 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-779a6d17-4ac3-4e20-abde-80fc7809dc40 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:04.928285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-779a6d17-4ac3-4e20-abde-80fc7809dc40 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:04.928975 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-779a6d17-4ac3-4e20-abde-80fc7809dc40 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:04.990832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:05.006698 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:05.016298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:05.024892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-da49b8fb-544b-40fd-8943-e03698bc34ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1338 microversion: 2.20 time: 0.272158 Oct 20 09:01:05.025981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 41/65] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:04 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1338 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:05.309859 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-779a6d17-4ac3-4e20-abde-80fc7809dc40 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:05.323355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5aa3793f-d97e-431d-b27d-178833a9c82f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:05.324912 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5aa3793f-d97e-431d-b27d-178833a9c82f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:05.325784 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-779a6d17-4ac3-4e20-abde-80fc7809dc40 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.425437 Oct 20 09:01:05.326580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 25/66] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:04 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:05.336584 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa3793f-d97e-431d-b27d-178833a9c82f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:05.339226 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa3793f-d97e-431d-b27d-178833a9c82f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:05.341681 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa3793f-d97e-431d-b27d-178833a9c82f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:05.342740 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa3793f-d97e-431d-b27d-178833a9c82f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:05.344304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa3793f-d97e-431d-b27d-178833a9c82f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:05.347704 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5aa3793f-d97e-431d-b27d-178833a9c82f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:05.579919 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5aa3793f-d97e-431d-b27d-178833a9c82f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:05.590758 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5aa3793f-d97e-431d-b27d-178833a9c82f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.276299 Oct 20 09:01:05.591936 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 42/67] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:05 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:05.777179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6ef0f45a-aa44-4e96-941b-b8709c8628c3 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:05.777802 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6ef0f45a-aa44-4e96-941b-b8709c8628c3 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:05.785244 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ef0f45a-aa44-4e96-941b-b8709c8628c3 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:05.788745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ef0f45a-aa44-4e96-941b-b8709c8628c3 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:05.789481 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ef0f45a-aa44-4e96-941b-b8709c8628c3 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:05.791538 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ef0f45a-aa44-4e96-941b-b8709c8628c3 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:05.793773 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ef0f45a-aa44-4e96-941b-b8709c8628c3 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:05.794614 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ef0f45a-aa44-4e96-941b-b8709c8628c3 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:06.052213 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:06.053812 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:06.063831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=26 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:06.068621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:06.070772 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:06.072825 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:06.074806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:06.075663 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:06.079006 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:06.366932 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=24 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:06.612735 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:06.627548 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:06.627548 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:06.627548 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f258c34b-ca9b-4f1e-8f8e-d35df196c5e1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.579555 Oct 20 09:01:06.636628 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 43/68] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:06 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:06.641386 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-13855155-4e1b-4b94-8e2b-01e011b62fe1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:06.641993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-13855155-4e1b-4b94-8e2b-01e011b62fe1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:06.659909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-13855155-4e1b-4b94-8e2b-01e011b62fe1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:06.673401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-13855155-4e1b-4b94-8e2b-01e011b62fe1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:06.674150 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-13855155-4e1b-4b94-8e2b-01e011b62fe1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:06.681191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6ef0f45a-aa44-4e96-941b-b8709c8628c3 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:06.694872 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-13855155-4e1b-4b94-8e2b-01e011b62fe1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:06.698307 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-13855155-4e1b-4b94-8e2b-01e011b62fe1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.021s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:06.698787 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-13855155-4e1b-4b94-8e2b-01e011b62fe1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:06.702413 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6ef0f45a-aa44-4e96-941b-b8709c8628c3 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.929397 Oct 20 09:01:06.715111 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 26/69] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:05 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 942 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:06.821649 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-f2b68d00-de4d-4742-8516-598afc29a8bb req-eb863e99-f8c4-48a4-b6fd-9e6577008956 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ec9ae57d-3e5a-490f-beba-d071e86a7b31", "tag": "7681f5a2-5dbc-4513-b44a-9c70525ecfee"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:06.836305 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-f2b68d00-de4d-4742-8516-598afc29a8bb req-eb863e99-f8c4-48a4-b6fd-9e6577008956 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:06.836786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-f2b68d00-de4d-4742-8516-598afc29a8bb req-eb863e99-f8c4-48a4-b6fd-9e6577008956 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:06.837277 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-f2b68d00-de4d-4742-8516-598afc29a8bb req-eb863e99-f8c4-48a4-b6fd-9e6577008956 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:06.860869 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-f2b68d00-de4d-4742-8516-598afc29a8bb req-eb863e99-f8c4-48a4-b6fd-9e6577008956 service nova] Creating event network-changed:7681f5a2-5dbc-4513-b44a-9c70525ecfee for instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 on hv-cinder-maste Oct 20 09:01:06.888509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-f2b68d00-de4d-4742-8516-598afc29a8bb req-eb863e99-f8c4-48a4-b6fd-9e6577008956 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.171844 Oct 20 09:01:06.889137 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 27/70] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:06.900981 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1e04096f-bac7-4b70-98de-d301d44f84f1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:06.911187 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1e04096f-bac7-4b70-98de-d301d44f84f1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:06.925082 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e04096f-bac7-4b70-98de-d301d44f84f1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:06.926150 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e04096f-bac7-4b70-98de-d301d44f84f1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:06.926825 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e04096f-bac7-4b70-98de-d301d44f84f1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:06.927450 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e04096f-bac7-4b70-98de-d301d44f84f1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:06.928152 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e04096f-bac7-4b70-98de-d301d44f84f1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:06.928714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e04096f-bac7-4b70-98de-d301d44f84f1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:07.015968 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-13855155-4e1b-4b94-8e2b-01e011b62fe1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:07.028081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-13855155-4e1b-4b94-8e2b-01e011b62fe1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.390307 Oct 20 09:01:07.031972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 44/71] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:06 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:07.041833 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-356dc000-0d75-478e-bc2d-d9448925e77c req-877e7d42-354f-4af4-94c2-03fbfbaf5463 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0dd1dfdc-5e1e-4a08-a814-4f783857b2f7", "tag": "0d706cc3-dd7f-4052-9a1e-5635f7716b26"}, {"name": "network-changed", "server_uuid": "91bccd9d-c1c7-477f-b8a5-914edd9fa2eb", "tag": "b5692122-4272-4400-bc7a-9069ae5bc48a"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:07.086143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-356dc000-0d75-478e-bc2d-d9448925e77c req-877e7d42-354f-4af4-94c2-03fbfbaf5463 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:07.086657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-356dc000-0d75-478e-bc2d-d9448925e77c req-877e7d42-354f-4af4-94c2-03fbfbaf5463 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:07.087097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-356dc000-0d75-478e-bc2d-d9448925e77c req-877e7d42-354f-4af4-94c2-03fbfbaf5463 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:07.151982 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-356dc000-0d75-478e-bc2d-d9448925e77c req-877e7d42-354f-4af4-94c2-03fbfbaf5463 service nova] Creating event network-changed:0d706cc3-dd7f-4052-9a1e-5635f7716b26 for instance 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 on hv-cinder-maste Oct 20 09:01:07.153000 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-356dc000-0d75-478e-bc2d-d9448925e77c req-877e7d42-354f-4af4-94c2-03fbfbaf5463 service nova] Creating event network-changed:b5692122-4272-4400-bc7a-9069ae5bc48a for instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb on hv-cinder-maste Oct 20 09:01:07.154039 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-356dc000-0d75-478e-bc2d-d9448925e77c req-877e7d42-354f-4af4-94c2-03fbfbaf5463 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=119067) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 20 09:01:07.163645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-356dc000-0d75-478e-bc2d-d9448925e77c req-877e7d42-354f-4af4-94c2-03fbfbaf5463 service nova] Acquired lock "compute-rpcapi-router" {{(pid=119067) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 20 09:01:07.170126 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a03967c-b241-476f-b854-7c307b6b4956 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:07.176817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a03967c-b241-476f-b854-7c307b6b4956 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:07.178172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a03967c-b241-476f-b854-7c307b6b4956 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:07.180222 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1e04096f-bac7-4b70-98de-d301d44f84f1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:07.181968 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a03967c-b241-476f-b854-7c307b6b4956 None None] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:07.182738 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a03967c-b241-476f-b854-7c307b6b4956 None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:07.183741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a03967c-b241-476f-b854-7c307b6b4956 None None] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:07.193062 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1e04096f-bac7-4b70-98de-d301d44f84f1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.302714 Oct 20 09:01:07.196068 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 28/72] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:06 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:07.204030 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.compute.rpcapi [None req-7a03967c-b241-476f-b854-7c307b6b4956 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 20 09:01:07.205235 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a03967c-b241-476f-b854-7c307b6b4956 None None] Releasing lock "compute-rpcapi-router" {{(pid=119067) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 20 09:01:07.231514 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7a03967c-b241-476f-b854-7c307b6b4956 None None] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.196777 Oct 20 09:01:07.233512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 45/73] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:07 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:07.657442 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:07.657954 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:07.666406 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:07.666958 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:07.667814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:07.668891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:07.669340 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:07.670272 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:07.731510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-92163cb9-4df1-48f9-be5e-53e560ff1fef tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:07.732789 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-92163cb9-4df1-48f9-be5e-53e560ff1fef tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:07.743555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92163cb9-4df1-48f9-be5e-53e560ff1fef tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:07.744819 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92163cb9-4df1-48f9-be5e-53e560ff1fef tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:07.745475 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92163cb9-4df1-48f9-be5e-53e560ff1fef tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:07.746346 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92163cb9-4df1-48f9-be5e-53e560ff1fef tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:07.746936 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92163cb9-4df1-48f9-be5e-53e560ff1fef tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:07.747220 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92163cb9-4df1-48f9-be5e-53e560ff1fef tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:07.868205 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:07.878898 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:07.881278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:07.883308 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b0105920-6108-4b9e-b57e-65370768968d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.229083 Oct 20 09:01:07.884186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 29/74] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:07 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:07.949647 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-92163cb9-4df1-48f9-be5e-53e560ff1fef tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:07.961733 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-92163cb9-4df1-48f9-be5e-53e560ff1fef tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.235470 Oct 20 09:01:07.966674 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 46/75] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:07 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:08.054884 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8d619ae0-439b-4c30-8019-4f24d505a3ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:08.055516 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8d619ae0-439b-4c30-8019-4f24d505a3ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:08.062857 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=78,nova_api:INSERT=7 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:08.065522 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d619ae0-439b-4c30-8019-4f24d505a3ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:08.066283 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d619ae0-439b-4c30-8019-4f24d505a3ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:08.066832 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d619ae0-439b-4c30-8019-4f24d505a3ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:08.067699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d619ae0-439b-4c30-8019-4f24d505a3ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:08.068482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d619ae0-439b-4c30-8019-4f24d505a3ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:08.069096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d619ae0-439b-4c30-8019-4f24d505a3ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:08.075784 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=65 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:08.219824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f3a53cd2-ad64-49b3-a96a-8d19abc38d2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:08.220634 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f3a53cd2-ad64-49b3-a96a-8d19abc38d2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:08.225197 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8d619ae0-439b-4c30-8019-4f24d505a3ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:08.230666 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3a53cd2-ad64-49b3-a96a-8d19abc38d2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:08.234560 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3a53cd2-ad64-49b3-a96a-8d19abc38d2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:08.234692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8d619ae0-439b-4c30-8019-4f24d505a3ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.184983 Oct 20 09:01:08.237810 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3a53cd2-ad64-49b3-a96a-8d19abc38d2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:08.240445 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3a53cd2-ad64-49b3-a96a-8d19abc38d2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:08.242702 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3a53cd2-ad64-49b3-a96a-8d19abc38d2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:08.243388 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3a53cd2-ad64-49b3-a96a-8d19abc38d2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:08.250328 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 30/76] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:08 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:08.394000 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f3a53cd2-ad64-49b3-a96a-8d19abc38d2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:08.404719 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f3a53cd2-ad64-49b3-a96a-8d19abc38d2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.190632 Oct 20 09:01:08.405921 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 47/77] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:08 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:08.908988 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-61667090-3687-41fc-bdf5-0bd50c42164c req-7e724f9c-f4cd-4a5e-a234-767d1e15f8e0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f8d61907-24ca-4e74-917e-d218d0f89557", "tag": "6ba2009e-08be-4c65-a1d5-5e9548c8d407"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:08.912216 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:08.913870 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:08.922265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-61667090-3687-41fc-bdf5-0bd50c42164c req-7e724f9c-f4cd-4a5e-a234-767d1e15f8e0 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:08.923550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-61667090-3687-41fc-bdf5-0bd50c42164c req-7e724f9c-f4cd-4a5e-a234-767d1e15f8e0 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:08.923930 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:08.924048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-61667090-3687-41fc-bdf5-0bd50c42164c req-7e724f9c-f4cd-4a5e-a234-767d1e15f8e0 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:08.925415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:08.926090 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:08.927633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:08.929138 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:08.931258 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:08.938562 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-61667090-3687-41fc-bdf5-0bd50c42164c req-7e724f9c-f4cd-4a5e-a234-767d1e15f8e0 service nova] Creating event network-changed:6ba2009e-08be-4c65-a1d5-5e9548c8d407 for instance f8d61907-24ca-4e74-917e-d218d0f89557 on hv-cinder-maste Oct 20 09:01:08.945119 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-61667090-3687-41fc-bdf5-0bd50c42164c req-7e724f9c-f4cd-4a5e-a234-767d1e15f8e0 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039407 Oct 20 09:01:08.946816 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 31/78] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:08.991913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4b14d648-7b08-44d1-9408-b53e492fbd9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:08.993591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4b14d648-7b08-44d1-9408-b53e492fbd9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:09.001344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4b14d648-7b08-44d1-9408-b53e492fbd9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:09.002299 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4b14d648-7b08-44d1-9408-b53e492fbd9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:09.003033 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4b14d648-7b08-44d1-9408-b53e492fbd9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:09.003928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4b14d648-7b08-44d1-9408-b53e492fbd9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:09.004583 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4b14d648-7b08-44d1-9408-b53e492fbd9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:09.005198 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4b14d648-7b08-44d1-9408-b53e492fbd9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:09.086450 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:09.095484 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:09.097759 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:09.100014 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7637363f-fea3-4a4c-b24d-584237094df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.194148 Oct 20 09:01:09.101058 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 48/79] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:08 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:01:09.255030 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4b14d648-7b08-44d1-9408-b53e492fbd9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:09.258870 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8fe75dc0-13b2-4999-a0e8-096889604d20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:09.259482 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8fe75dc0-13b2-4999-a0e8-096889604d20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:09.265781 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8fe75dc0-13b2-4999-a0e8-096889604d20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:09.266339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8fe75dc0-13b2-4999-a0e8-096889604d20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:09.266452 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4b14d648-7b08-44d1-9408-b53e492fbd9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.282236 Oct 20 09:01:09.266871 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8fe75dc0-13b2-4999-a0e8-096889604d20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:09.267556 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8fe75dc0-13b2-4999-a0e8-096889604d20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:09.268002 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8fe75dc0-13b2-4999-a0e8-096889604d20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:09.268440 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8fe75dc0-13b2-4999-a0e8-096889604d20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:09.270513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 32/80] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:08 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:09.429442 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-835746ec-ab22-43db-a2c1-a2b1d63741e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:09.430339 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-835746ec-ab22-43db-a2c1-a2b1d63741e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:09.439640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8fe75dc0-13b2-4999-a0e8-096889604d20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:09.448142 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8fe75dc0-13b2-4999-a0e8-096889604d20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.193124 Oct 20 09:01:09.467045 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 49/81] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:09 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:09.467457 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-835746ec-ab22-43db-a2c1-a2b1d63741e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:09.467457 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-835746ec-ab22-43db-a2c1-a2b1d63741e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:09.467457 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-835746ec-ab22-43db-a2c1-a2b1d63741e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:09.467457 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-835746ec-ab22-43db-a2c1-a2b1d63741e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:09.467721 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-835746ec-ab22-43db-a2c1-a2b1d63741e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:09.467721 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-835746ec-ab22-43db-a2c1-a2b1d63741e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:09.614497 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-835746ec-ab22-43db-a2c1-a2b1d63741e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:09.622714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-835746ec-ab22-43db-a2c1-a2b1d63741e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.197848 Oct 20 09:01:09.624372 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 33/82] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:09 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:10.130018 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:10.130667 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:10.149965 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:10.150604 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:10.151130 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:10.151844 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:10.152451 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:10.153141 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:10.294682 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d3cc555d-a7f0-4dea-aa5a-9df188556f89 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:10.296409 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d3cc555d-a7f0-4dea-aa5a-9df188556f89 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:10.311278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d3cc555d-a7f0-4dea-aa5a-9df188556f89 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:10.313215 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d3cc555d-a7f0-4dea-aa5a-9df188556f89 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:10.314191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d3cc555d-a7f0-4dea-aa5a-9df188556f89 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:10.315380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d3cc555d-a7f0-4dea-aa5a-9df188556f89 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:10.316053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d3cc555d-a7f0-4dea-aa5a-9df188556f89 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:10.318027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d3cc555d-a7f0-4dea-aa5a-9df188556f89 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:10.507389 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:10.517025 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:10.518503 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:10.519620 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8aca8cdd-bb40-4311-adc6-9e08c2ef3df6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.395696 Oct 20 09:01:10.522046 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 50/83] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:10 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:10.531784 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-348f39a2-c864-45ba-b343-94b66ead7f03 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:10.532525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-348f39a2-c864-45ba-b343-94b66ead7f03 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:10.537328 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d3cc555d-a7f0-4dea-aa5a-9df188556f89 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:10.540880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-348f39a2-c864-45ba-b343-94b66ead7f03 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:10.541582 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-348f39a2-c864-45ba-b343-94b66ead7f03 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:10.542197 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-348f39a2-c864-45ba-b343-94b66ead7f03 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:10.545913 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-348f39a2-c864-45ba-b343-94b66ead7f03 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:10.546613 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-348f39a2-c864-45ba-b343-94b66ead7f03 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:10.547042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-348f39a2-c864-45ba-b343-94b66ead7f03 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:10.549420 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d3cc555d-a7f0-4dea-aa5a-9df188556f89 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.260175 Oct 20 09:01:10.549994 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 34/84] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:10 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:10.649818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e3f27b65-7ac9-4985-b8cb-b7e3d6515945 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:10.650521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e3f27b65-7ac9-4985-b8cb-b7e3d6515945 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:10.658404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3f27b65-7ac9-4985-b8cb-b7e3d6515945 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:10.659358 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3f27b65-7ac9-4985-b8cb-b7e3d6515945 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:10.660159 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3f27b65-7ac9-4985-b8cb-b7e3d6515945 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:10.660858 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3f27b65-7ac9-4985-b8cb-b7e3d6515945 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:10.661477 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3f27b65-7ac9-4985-b8cb-b7e3d6515945 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:10.662351 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3f27b65-7ac9-4985-b8cb-b7e3d6515945 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:10.717258 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-348f39a2-c864-45ba-b343-94b66ead7f03 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:10.725352 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-348f39a2-c864-45ba-b343-94b66ead7f03 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.202666 Oct 20 09:01:10.725901 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 51/85] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:10 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:10.871343 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e3f27b65-7ac9-4985-b8cb-b7e3d6515945 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:10.881594 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e3f27b65-7ac9-4985-b8cb-b7e3d6515945 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.236004 Oct 20 09:01:10.892113 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 35/86] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:10 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:11.545528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:11.546201 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:11.557073 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:11.557846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:11.558635 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:11.559661 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:11.560577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:11.561199 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:11.577394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d7703d49-6643-489b-85ac-2ad58823ece9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:11.578788 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d7703d49-6643-489b-85ac-2ad58823ece9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:11.597706 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7703d49-6643-489b-85ac-2ad58823ece9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:11.598639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7703d49-6643-489b-85ac-2ad58823ece9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:11.600225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7703d49-6643-489b-85ac-2ad58823ece9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:11.602806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7703d49-6643-489b-85ac-2ad58823ece9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:11.605444 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7703d49-6643-489b-85ac-2ad58823ece9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:11.606192 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7703d49-6643-489b-85ac-2ad58823ece9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:11.908230 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:11.920030 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:11.922042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:11.923408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f4c84b00-7f7e-411f-a103-0096c488f0b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.382070 Oct 20 09:01:11.924161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 52/87] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:11 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:11.935083 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2939c373-d2f6-4eb2-a562-89f4c670594b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:11.936041 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2939c373-d2f6-4eb2-a562-89f4c670594b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:11.944452 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2939c373-d2f6-4eb2-a562-89f4c670594b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:11.944962 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2939c373-d2f6-4eb2-a562-89f4c670594b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:11.945430 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2939c373-d2f6-4eb2-a562-89f4c670594b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:11.946362 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2939c373-d2f6-4eb2-a562-89f4c670594b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:11.946857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2939c373-d2f6-4eb2-a562-89f4c670594b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:11.947279 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2939c373-d2f6-4eb2-a562-89f4c670594b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:12.118134 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2939c373-d2f6-4eb2-a562-89f4c670594b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:12.120549 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d7703d49-6643-489b-85ac-2ad58823ece9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:12.128861 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d7703d49-6643-489b-85ac-2ad58823ece9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.557077 Oct 20 09:01:12.130674 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2939c373-d2f6-4eb2-a562-89f4c670594b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.204618 Oct 20 09:01:12.131262 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 53/88] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:11 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:12.132544 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 36/89] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:11 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:12.153497 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f5b9418e-8ac2-4793-a371-7ceacf038f7d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:12.154508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f5b9418e-8ac2-4793-a371-7ceacf038f7d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:12.162540 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5b9418e-8ac2-4793-a371-7ceacf038f7d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:12.163198 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5b9418e-8ac2-4793-a371-7ceacf038f7d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:12.163709 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5b9418e-8ac2-4793-a371-7ceacf038f7d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:12.165382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5b9418e-8ac2-4793-a371-7ceacf038f7d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:12.166105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5b9418e-8ac2-4793-a371-7ceacf038f7d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:12.166828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5b9418e-8ac2-4793-a371-7ceacf038f7d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:12.189357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=100 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:12.537996 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f5b9418e-8ac2-4793-a371-7ceacf038f7d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:12.548469 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f5b9418e-8ac2-4793-a371-7ceacf038f7d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.416167 Oct 20 09:01:12.549043 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 54/90] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:12 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:12.948901 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:12.949512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:12.955965 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:12.956482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:12.956915 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:12.957611 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:12.958060 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:12.958496 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:13.153475 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6e2f91f8-85ec-49bb-91df-47eda379966a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:13.161925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6e2f91f8-85ec-49bb-91df-47eda379966a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:13.171220 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6e2f91f8-85ec-49bb-91df-47eda379966a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:13.172924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6e2f91f8-85ec-49bb-91df-47eda379966a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:13.174716 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6e2f91f8-85ec-49bb-91df-47eda379966a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:13.175531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6e2f91f8-85ec-49bb-91df-47eda379966a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:13.175996 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6e2f91f8-85ec-49bb-91df-47eda379966a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:13.176419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6e2f91f8-85ec-49bb-91df-47eda379966a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:13.344860 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6e2f91f8-85ec-49bb-91df-47eda379966a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:13.345591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:13.353761 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6e2f91f8-85ec-49bb-91df-47eda379966a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.204093 Oct 20 09:01:13.354348 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 55/91] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:13 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:13.354920 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:13.357029 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:13.358564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-46554fe8-30c9-45d5-ac2e-fcee2b774c1a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.414363 Oct 20 09:01:13.359123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4e9ac375-7daf-425c-aaaf-d77184b8f779 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:13.359837 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 37/92] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:12 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:13.375231 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4e9ac375-7daf-425c-aaaf-d77184b8f779 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:13.384523 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e9ac375-7daf-425c-aaaf-d77184b8f779 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:13.385454 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e9ac375-7daf-425c-aaaf-d77184b8f779 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:13.385897 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e9ac375-7daf-425c-aaaf-d77184b8f779 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:13.386681 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e9ac375-7daf-425c-aaaf-d77184b8f779 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:13.387137 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e9ac375-7daf-425c-aaaf-d77184b8f779 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:13.387592 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e9ac375-7daf-425c-aaaf-d77184b8f779 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:13.533700 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4e9ac375-7daf-425c-aaaf-d77184b8f779 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:13.542574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4e9ac375-7daf-425c-aaaf-d77184b8f779 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.186892 Oct 20 09:01:13.550610 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 56/93] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:13 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:13.573282 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d51f7a95-519b-45e3-8f63-01be05ec36c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:13.574038 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d51f7a95-519b-45e3-8f63-01be05ec36c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:13.581410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51f7a95-519b-45e3-8f63-01be05ec36c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:13.581962 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51f7a95-519b-45e3-8f63-01be05ec36c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:13.583303 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51f7a95-519b-45e3-8f63-01be05ec36c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:13.584305 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51f7a95-519b-45e3-8f63-01be05ec36c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:13.585175 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51f7a95-519b-45e3-8f63-01be05ec36c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:13.586439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51f7a95-519b-45e3-8f63-01be05ec36c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:13.774311 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d51f7a95-519b-45e3-8f63-01be05ec36c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:13.785960 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d51f7a95-519b-45e3-8f63-01be05ec36c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.216713 Oct 20 09:01:13.786961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 38/94] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:13 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:14.379196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-145a0e4e-20ec-460d-bc07-b3bb89b69ee0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:14.380025 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-145a0e4e-20ec-460d-bc07-b3bb89b69ee0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:14.388926 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:14.393277 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:14.404734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-145a0e4e-20ec-460d-bc07-b3bb89b69ee0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:14.408146 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-145a0e4e-20ec-460d-bc07-b3bb89b69ee0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:14.409956 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-145a0e4e-20ec-460d-bc07-b3bb89b69ee0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:14.411430 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-145a0e4e-20ec-460d-bc07-b3bb89b69ee0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:14.413501 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-145a0e4e-20ec-460d-bc07-b3bb89b69ee0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:14.413816 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:14.414458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:14.415091 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-145a0e4e-20ec-460d-bc07-b3bb89b69ee0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:14.416239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:14.419699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:14.421216 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:14.421832 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:14.715550 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-145a0e4e-20ec-460d-bc07-b3bb89b69ee0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:14.724108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-145a0e4e-20ec-460d-bc07-b3bb89b69ee0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.348637 Oct 20 09:01:14.729309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 57/95] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:14 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:14.733271 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=4 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:14.735458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-332e0f8d-4a87-488d-abea-79da0ab49938 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:14.736281 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-332e0f8d-4a87-488d-abea-79da0ab49938 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:14.744067 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-332e0f8d-4a87-488d-abea-79da0ab49938 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:14.744543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-332e0f8d-4a87-488d-abea-79da0ab49938 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:14.745139 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-332e0f8d-4a87-488d-abea-79da0ab49938 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:14.745835 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-332e0f8d-4a87-488d-abea-79da0ab49938 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:14.746281 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-332e0f8d-4a87-488d-abea-79da0ab49938 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:14.747001 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-332e0f8d-4a87-488d-abea-79da0ab49938 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:14.967101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-332e0f8d-4a87-488d-abea-79da0ab49938 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:14.967269 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:14.977899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-332e0f8d-4a87-488d-abea-79da0ab49938 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.246917 Oct 20 09:01:14.979586 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 58/96] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:14 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:14.983465 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:14.984847 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-75c13a90-797b-4a46-81c2-626eedccd56d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:14.984969 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:14.985765 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-75c13a90-797b-4a46-81c2-626eedccd56d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:14.986916 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3bfc4a92-60b4-4019-9f07-b685c1571ff9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.602828 Oct 20 09:01:14.991798 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 39/97] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:14 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:14.993404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c13a90-797b-4a46-81c2-626eedccd56d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:14.994045 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c13a90-797b-4a46-81c2-626eedccd56d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:14.994732 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c13a90-797b-4a46-81c2-626eedccd56d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:14.995599 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c13a90-797b-4a46-81c2-626eedccd56d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:14.996198 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c13a90-797b-4a46-81c2-626eedccd56d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:14.996771 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c13a90-797b-4a46-81c2-626eedccd56d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:15.167931 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-75c13a90-797b-4a46-81c2-626eedccd56d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:15.175671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-75c13a90-797b-4a46-81c2-626eedccd56d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.194903 Oct 20 09:01:15.176442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 59/98] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:14 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:15.753688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-21c9da39-e936-4f83-af42-0c017097a540 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:15.754308 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-21c9da39-e936-4f83-af42-0c017097a540 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:15.765719 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21c9da39-e936-4f83-af42-0c017097a540 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:15.766427 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21c9da39-e936-4f83-af42-0c017097a540 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:15.766812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21c9da39-e936-4f83-af42-0c017097a540 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:15.767694 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21c9da39-e936-4f83-af42-0c017097a540 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:15.768492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21c9da39-e936-4f83-af42-0c017097a540 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:15.770452 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21c9da39-e936-4f83-af42-0c017097a540 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:16.004110 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-dde80c40-83d1-4139-ae0c-68d102e078f4 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:16.005516 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-dde80c40-83d1-4139-ae0c-68d102e078f4 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:16.028778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dde80c40-83d1-4139-ae0c-68d102e078f4 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:16.031304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dde80c40-83d1-4139-ae0c-68d102e078f4 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:16.031924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dde80c40-83d1-4139-ae0c-68d102e078f4 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:16.034371 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dde80c40-83d1-4139-ae0c-68d102e078f4 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:16.036603 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dde80c40-83d1-4139-ae0c-68d102e078f4 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:16.037262 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dde80c40-83d1-4139-ae0c-68d102e078f4 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:16.325774 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-21c9da39-e936-4f83-af42-0c017097a540 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:16.351785 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-21c9da39-e936-4f83-af42-0c017097a540 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.601625 Oct 20 09:01:16.352434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 40/99] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:15 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:16.362913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:16.363907 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:16.380004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:16.381918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:16.383653 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:16.387356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:16.388014 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:16.389116 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:16.446744 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-dde80c40-83d1-4139-ae0c-68d102e078f4 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:16.464222 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-dde80c40-83d1-4139-ae0c-68d102e078f4 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.458907 Oct 20 09:01:16.464842 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 60/100] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:15 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:16.474434 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2b0611f2-943a-4b85-917e-3401d38c3b2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:16.485804 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2b0611f2-943a-4b85-917e-3401d38c3b2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:16.501101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2b0611f2-943a-4b85-917e-3401d38c3b2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:16.502254 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2b0611f2-943a-4b85-917e-3401d38c3b2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:16.504064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2b0611f2-943a-4b85-917e-3401d38c3b2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:16.505113 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2b0611f2-943a-4b85-917e-3401d38c3b2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:16.509350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2b0611f2-943a-4b85-917e-3401d38c3b2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:16.510004 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2b0611f2-943a-4b85-917e-3401d38c3b2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:16.823652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2b0611f2-943a-4b85-917e-3401d38c3b2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:16.839310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2b0611f2-943a-4b85-917e-3401d38c3b2d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.368516 Oct 20 09:01:16.842831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 61/101] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:16 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:16.868577 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:16.881319 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:16.884161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:16.886696 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-225fe409-6e73-4fc2-bcba-0565eaa36933 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.533135 Oct 20 09:01:16.887667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 41/102] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:16 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:17.002328 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-e221d149-ee4d-4211-a7c4-fa17cd91cedb req-eb9ba312-484f-495e-b61c-643666952aff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cabc1ab-03aa-4ff2-be06-d99f9efc6150", "name": "network-vif-plugged", "status": "completed", "tag": "761aed46-611d-4ca6-9fc5-7334f3e86a8f"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:17.028573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-e221d149-ee4d-4211-a7c4-fa17cd91cedb req-eb9ba312-484f-495e-b61c-643666952aff service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:17.029922 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-e221d149-ee4d-4211-a7c4-fa17cd91cedb req-eb9ba312-484f-495e-b61c-643666952aff service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:17.035359 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-e221d149-ee4d-4211-a7c4-fa17cd91cedb req-eb9ba312-484f-495e-b61c-643666952aff service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:17.062175 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-e221d149-ee4d-4211-a7c4-fa17cd91cedb req-eb9ba312-484f-495e-b61c-643666952aff service nova] Creating event network-vif-plugged:761aed46-611d-4ca6-9fc5-7334f3e86a8f for instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 on hv-cinder-maste Oct 20 09:01:17.068598 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-e221d149-ee4d-4211-a7c4-fa17cd91cedb req-eb9ba312-484f-495e-b61c-643666952aff service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.223718 Oct 20 09:01:17.069733 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 62/103] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:17.388414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-185f360a-99fc-4210-8a8c-ee9cc1eb08d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:17.389202 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-185f360a-99fc-4210-8a8c-ee9cc1eb08d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:17.396305 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-185f360a-99fc-4210-8a8c-ee9cc1eb08d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:17.397091 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-185f360a-99fc-4210-8a8c-ee9cc1eb08d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:17.397775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-185f360a-99fc-4210-8a8c-ee9cc1eb08d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:17.398958 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-185f360a-99fc-4210-8a8c-ee9cc1eb08d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:17.399634 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-185f360a-99fc-4210-8a8c-ee9cc1eb08d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:17.400254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-185f360a-99fc-4210-8a8c-ee9cc1eb08d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:17.485998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:17.490860 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-80fd0d84-d975-499c-b345-ef4cffeaab10 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:17.491568 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-80fd0d84-d975-499c-b345-ef4cffeaab10 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:17.505546 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fd0d84-d975-499c-b345-ef4cffeaab10 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:17.510991 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fd0d84-d975-499c-b345-ef4cffeaab10 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:17.511446 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fd0d84-d975-499c-b345-ef4cffeaab10 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:17.514082 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fd0d84-d975-499c-b345-ef4cffeaab10 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:17.515804 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fd0d84-d975-499c-b345-ef4cffeaab10 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:17.519041 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fd0d84-d975-499c-b345-ef4cffeaab10 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:17.623131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-185f360a-99fc-4210-8a8c-ee9cc1eb08d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:17.634211 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-185f360a-99fc-4210-8a8c-ee9cc1eb08d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.250774 Oct 20 09:01:17.646659 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 42/104] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:17 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:17.750652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-80fd0d84-d975-499c-b345-ef4cffeaab10 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:17.759430 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-80fd0d84-d975-499c-b345-ef4cffeaab10 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.274587 Oct 20 09:01:17.760738 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 63/105] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:17 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:17.867419 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d941cfe1-23b0-4f09-a8a2-ea2bf9b7a792 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:17.868361 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d941cfe1-23b0-4f09-a8a2-ea2bf9b7a792 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:17.875553 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d941cfe1-23b0-4f09-a8a2-ea2bf9b7a792 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:17.876191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d941cfe1-23b0-4f09-a8a2-ea2bf9b7a792 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:17.876818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d941cfe1-23b0-4f09-a8a2-ea2bf9b7a792 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:17.877561 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d941cfe1-23b0-4f09-a8a2-ea2bf9b7a792 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:17.878322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d941cfe1-23b0-4f09-a8a2-ea2bf9b7a792 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:17.878972 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d941cfe1-23b0-4f09-a8a2-ea2bf9b7a792 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:17.914327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:17.915311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:17.924408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:17.926272 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:17.926944 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:17.928077 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:17.928972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:17.930003 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:18.080040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d941cfe1-23b0-4f09-a8a2-ea2bf9b7a792 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:18.088709 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d941cfe1-23b0-4f09-a8a2-ea2bf9b7a792 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.225877 Oct 20 09:01:18.089429 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 43/106] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:17 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:18.133284 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:18.141528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:18.142937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:18.144048 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6d591e7a-3801-4b4b-8ec5-d69eafc358a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.234730 Oct 20 09:01:18.144622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 64/107] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:17 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:01:18.673362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4bc1823a-05c1-4630-862d-21e563fa3d00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:18.673971 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4bc1823a-05c1-4630-862d-21e563fa3d00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:18.680710 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bc1823a-05c1-4630-862d-21e563fa3d00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:18.681502 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bc1823a-05c1-4630-862d-21e563fa3d00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:18.681993 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bc1823a-05c1-4630-862d-21e563fa3d00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:18.682786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bc1823a-05c1-4630-862d-21e563fa3d00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:18.683444 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bc1823a-05c1-4630-862d-21e563fa3d00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:18.684051 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bc1823a-05c1-4630-862d-21e563fa3d00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:18.803292 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7a5712d2-5a18-4b7d-b62d-ed33c2f56522 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:18.803930 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7a5712d2-5a18-4b7d-b62d-ed33c2f56522 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:18.815457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a5712d2-5a18-4b7d-b62d-ed33c2f56522 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:18.817859 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a5712d2-5a18-4b7d-b62d-ed33c2f56522 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:18.818342 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a5712d2-5a18-4b7d-b62d-ed33c2f56522 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:18.819213 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a5712d2-5a18-4b7d-b62d-ed33c2f56522 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:18.819665 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a5712d2-5a18-4b7d-b62d-ed33c2f56522 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:18.822189 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a5712d2-5a18-4b7d-b62d-ed33c2f56522 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:18.888263 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4bc1823a-05c1-4630-862d-21e563fa3d00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:18.904061 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4bc1823a-05c1-4630-862d-21e563fa3d00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.234629 Oct 20 09:01:18.904876 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 44/108] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:18 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:19.018437 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7a5712d2-5a18-4b7d-b62d-ed33c2f56522 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:19.033924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7a5712d2-5a18-4b7d-b62d-ed33c2f56522 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.249799 Oct 20 09:01:19.035167 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 65/109] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:18 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:19.100834 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-06b5dda9-37b7-4164-b0df-dd39df8c7768 req-19a32c85-f7c4-4ce6-9aa1-a6c81c0d0719 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0dd1dfdc-5e1e-4a08-a814-4f783857b2f7", "name": "network-vif-plugged", "status": "completed", "tag": "0d706cc3-dd7f-4052-9a1e-5635f7716b26"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:19.117461 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f518a20a-29a8-47a5-b224-e83b2154508e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:19.118078 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f518a20a-29a8-47a5-b224-e83b2154508e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:19.127923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f518a20a-29a8-47a5-b224-e83b2154508e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:19.128431 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f518a20a-29a8-47a5-b224-e83b2154508e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:19.128855 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f518a20a-29a8-47a5-b224-e83b2154508e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:19.129643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f518a20a-29a8-47a5-b224-e83b2154508e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:19.130185 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f518a20a-29a8-47a5-b224-e83b2154508e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:19.132673 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f518a20a-29a8-47a5-b224-e83b2154508e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:19.135963 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-06b5dda9-37b7-4164-b0df-dd39df8c7768 req-19a32c85-f7c4-4ce6-9aa1-a6c81c0d0719 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:19.136455 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-06b5dda9-37b7-4164-b0df-dd39df8c7768 req-19a32c85-f7c4-4ce6-9aa1-a6c81c0d0719 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:19.139067 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-06b5dda9-37b7-4164-b0df-dd39df8c7768 req-19a32c85-f7c4-4ce6-9aa1-a6c81c0d0719 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:19.171710 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-06b5dda9-37b7-4164-b0df-dd39df8c7768 req-19a32c85-f7c4-4ce6-9aa1-a6c81c0d0719 service nova] Creating event network-vif-plugged:0d706cc3-dd7f-4052-9a1e-5635f7716b26 for instance 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 on hv-cinder-maste Oct 20 09:01:19.186555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-06b5dda9-37b7-4164-b0df-dd39df8c7768 req-19a32c85-f7c4-4ce6-9aa1-a6c81c0d0719 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089807 Oct 20 09:01:19.187189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 45/110] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:19.196816 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:19.197531 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:19.212872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:19.213520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:19.213963 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:19.214760 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:19.216459 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:19.217044 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:19.311466 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f518a20a-29a8-47a5-b224-e83b2154508e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:19.323006 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f518a20a-29a8-47a5-b224-e83b2154508e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.216057 Oct 20 09:01:19.323430 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 66/111] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:19 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:19.871041 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:19.887905 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:19.889653 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:19.891783 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7b3531ae-54c2-4ea2-b2a6-74199192b5e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.701994 Oct 20 09:01:19.916009 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 46/112] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:19 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:19.932461 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4a2d6900-5037-4a43-9fab-bf4b78c562fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:19.939281 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4a2d6900-5037-4a43-9fab-bf4b78c562fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:19.949853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a2d6900-5037-4a43-9fab-bf4b78c562fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:19.950850 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a2d6900-5037-4a43-9fab-bf4b78c562fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:19.951515 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a2d6900-5037-4a43-9fab-bf4b78c562fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:19.952751 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a2d6900-5037-4a43-9fab-bf4b78c562fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:19.954217 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a2d6900-5037-4a43-9fab-bf4b78c562fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:19.954695 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a2d6900-5037-4a43-9fab-bf4b78c562fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:20.069870 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6256fcd5-9ba2-4cf5-88bc-c07ebef5ebed tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:20.072666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6256fcd5-9ba2-4cf5-88bc-c07ebef5ebed tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:20.081753 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6256fcd5-9ba2-4cf5-88bc-c07ebef5ebed tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:20.083138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6256fcd5-9ba2-4cf5-88bc-c07ebef5ebed tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:20.083893 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6256fcd5-9ba2-4cf5-88bc-c07ebef5ebed tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:20.086927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6256fcd5-9ba2-4cf5-88bc-c07ebef5ebed tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:20.092020 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6256fcd5-9ba2-4cf5-88bc-c07ebef5ebed tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:20.092618 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6256fcd5-9ba2-4cf5-88bc-c07ebef5ebed tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:20.167667 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4a2d6900-5037-4a43-9fab-bf4b78c562fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:20.177728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4a2d6900-5037-4a43-9fab-bf4b78c562fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.253161 Oct 20 09:01:20.178303 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 67/113] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:19 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:20.309510 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6256fcd5-9ba2-4cf5-88bc-c07ebef5ebed tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:20.321048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6256fcd5-9ba2-4cf5-88bc-c07ebef5ebed tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.263843 Oct 20 09:01:20.333473 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 47/114] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:20 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:20.346680 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0bf72e57-7d0b-437d-a6e1-68dea5f119c9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:20.347266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0bf72e57-7d0b-437d-a6e1-68dea5f119c9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:20.355823 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf72e57-7d0b-437d-a6e1-68dea5f119c9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:20.356583 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf72e57-7d0b-437d-a6e1-68dea5f119c9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:20.356977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf72e57-7d0b-437d-a6e1-68dea5f119c9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:20.357675 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf72e57-7d0b-437d-a6e1-68dea5f119c9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:20.358136 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf72e57-7d0b-437d-a6e1-68dea5f119c9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:20.358554 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf72e57-7d0b-437d-a6e1-68dea5f119c9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:20.525555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0bf72e57-7d0b-437d-a6e1-68dea5f119c9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:20.533236 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0bf72e57-7d0b-437d-a6e1-68dea5f119c9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.190339 Oct 20 09:01:20.534132 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 68/115] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:20 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:20.924393 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:20.925710 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:20.933090 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:20.933892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:20.937023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:20.938130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:20.938995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:20.939729 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:20.971875 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:21.095015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:21.105192 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:21.107014 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:21.108118 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-99723067-99ac-4d83-be54-f413f1693cbd tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.189157 Oct 20 09:01:21.108687 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 48/116] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:20 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:21.201297 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-92abf611-fb5f-4f8e-b800-03f8323b0d4d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:21.201901 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-92abf611-fb5f-4f8e-b800-03f8323b0d4d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:21.212392 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92abf611-fb5f-4f8e-b800-03f8323b0d4d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:21.212963 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92abf611-fb5f-4f8e-b800-03f8323b0d4d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:21.213406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92abf611-fb5f-4f8e-b800-03f8323b0d4d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:21.214084 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92abf611-fb5f-4f8e-b800-03f8323b0d4d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:21.214735 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92abf611-fb5f-4f8e-b800-03f8323b0d4d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:21.215858 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92abf611-fb5f-4f8e-b800-03f8323b0d4d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:21.230279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-3a199ef0-bf59-4f01-84cc-7f79420413d8 req-d8e0acad-d73e-42f8-8b87-18255304892f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91bccd9d-c1c7-477f-b8a5-914edd9fa2eb", "name": "network-vif-plugged", "status": "completed", "tag": "b5692122-4272-4400-bc7a-9069ae5bc48a"}, {"server_uuid": "ec9ae57d-3e5a-490f-beba-d071e86a7b31", "name": "network-vif-plugged", "status": "completed", "tag": "7681f5a2-5dbc-4513-b44a-9c70525ecfee"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:21.268649 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-3a199ef0-bf59-4f01-84cc-7f79420413d8 req-d8e0acad-d73e-42f8-8b87-18255304892f service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:21.271107 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-3a199ef0-bf59-4f01-84cc-7f79420413d8 req-d8e0acad-d73e-42f8-8b87-18255304892f service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:21.271914 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-3a199ef0-bf59-4f01-84cc-7f79420413d8 req-d8e0acad-d73e-42f8-8b87-18255304892f service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:21.293866 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-3a199ef0-bf59-4f01-84cc-7f79420413d8 req-d8e0acad-d73e-42f8-8b87-18255304892f service nova] Creating event network-vif-plugged:b5692122-4272-4400-bc7a-9069ae5bc48a for instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb on hv-cinder-maste Oct 20 09:01:21.294597 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-3a199ef0-bf59-4f01-84cc-7f79420413d8 req-d8e0acad-d73e-42f8-8b87-18255304892f service nova] Creating event network-vif-plugged:7681f5a2-5dbc-4513-b44a-9c70525ecfee for instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 on hv-cinder-maste Oct 20 09:01:21.304919 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-3a199ef0-bf59-4f01-84cc-7f79420413d8 req-d8e0acad-d73e-42f8-8b87-18255304892f service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.080321 Oct 20 09:01:21.309926 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 49/117] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:21 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:21.350646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e57aca75-f42e-4edb-8dfb-0565b5eaf7a9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:21.351467 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e57aca75-f42e-4edb-8dfb-0565b5eaf7a9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:21.366439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e57aca75-f42e-4edb-8dfb-0565b5eaf7a9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:21.367716 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e57aca75-f42e-4edb-8dfb-0565b5eaf7a9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:21.368313 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e57aca75-f42e-4edb-8dfb-0565b5eaf7a9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:21.369538 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e57aca75-f42e-4edb-8dfb-0565b5eaf7a9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:21.371616 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e57aca75-f42e-4edb-8dfb-0565b5eaf7a9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:21.375475 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e57aca75-f42e-4edb-8dfb-0565b5eaf7a9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:21.430922 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-92abf611-fb5f-4f8e-b800-03f8323b0d4d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:21.440461 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-92abf611-fb5f-4f8e-b800-03f8323b0d4d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1321 microversion: 2.1 time: 0.242829 Oct 20 09:01:21.458512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 69/118] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:21 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1321 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:21.562061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-cec2c693-64c7-4253-94f0-cc475a6ad497 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:21.563239 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-cec2c693-64c7-4253-94f0-cc475a6ad497 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:21.580094 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cec2c693-64c7-4253-94f0-cc475a6ad497 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:21.580976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cec2c693-64c7-4253-94f0-cc475a6ad497 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:21.582296 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cec2c693-64c7-4253-94f0-cc475a6ad497 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:21.583507 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cec2c693-64c7-4253-94f0-cc475a6ad497 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:21.587216 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cec2c693-64c7-4253-94f0-cc475a6ad497 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:21.589468 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cec2c693-64c7-4253-94f0-cc475a6ad497 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:21.638639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e57aca75-f42e-4edb-8dfb-0565b5eaf7a9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:21.648647 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e57aca75-f42e-4edb-8dfb-0565b5eaf7a9 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.303345 Oct 20 09:01:21.651946 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 50/119] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:21 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:21.863646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-cec2c693-64c7-4253-94f0-cc475a6ad497 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:21.872987 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-cec2c693-64c7-4253-94f0-cc475a6ad497 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.316538 Oct 20 09:01:21.874892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 70/120] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:21 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:22.139315 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:22.140552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:22.147410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.148334 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.149062 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.150232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.150954 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.151514 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.373160 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:22.381557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:22.385461 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:22.386797 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-333bfc9b-b9e5-47b2-a372-67149e7ea6b6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.255198 Oct 20 09:01:22.387925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 51/121] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:22 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:22.465208 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-755358ed-2471-4b5f-ac32-2cbb561820fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:22.465755 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-755358ed-2471-4b5f-ac32-2cbb561820fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:22.472179 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-755358ed-2471-4b5f-ac32-2cbb561820fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.472649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-755358ed-2471-4b5f-ac32-2cbb561820fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.473376 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-755358ed-2471-4b5f-ac32-2cbb561820fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.474253 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-755358ed-2471-4b5f-ac32-2cbb561820fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.475376 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-755358ed-2471-4b5f-ac32-2cbb561820fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.476694 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-755358ed-2471-4b5f-ac32-2cbb561820fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.619983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-755358ed-2471-4b5f-ac32-2cbb561820fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:22.624838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=100 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:22.632277 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-755358ed-2471-4b5f-ac32-2cbb561820fe tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1514 microversion: 2.1 time: 0.171516 Oct 20 09:01:22.633439 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 71/122] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:22 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1514 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:22.656192 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f09fb4ca-9cc3-48c0-9bc7-c472663b71ea tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:22.658814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f09fb4ca-9cc3-48c0-9bc7-c472663b71ea tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:22.665485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f09fb4ca-9cc3-48c0-9bc7-c472663b71ea tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.666262 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f09fb4ca-9cc3-48c0-9bc7-c472663b71ea tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.666918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f09fb4ca-9cc3-48c0-9bc7-c472663b71ea tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.667854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f09fb4ca-9cc3-48c0-9bc7-c472663b71ea tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.668384 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f09fb4ca-9cc3-48c0-9bc7-c472663b71ea tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.668972 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f09fb4ca-9cc3-48c0-9bc7-c472663b71ea tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.676079 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-88f13359-3ad6-40c9-9b93-fc24e234b4ae tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:22.676763 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-88f13359-3ad6-40c9-9b93-fc24e234b4ae tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:22.683467 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-88f13359-3ad6-40c9-9b93-fc24e234b4ae tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.683944 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-88f13359-3ad6-40c9-9b93-fc24e234b4ae tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.684373 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-88f13359-3ad6-40c9-9b93-fc24e234b4ae tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.685052 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-88f13359-3ad6-40c9-9b93-fc24e234b4ae tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.685492 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-88f13359-3ad6-40c9-9b93-fc24e234b4ae tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.685908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-88f13359-3ad6-40c9-9b93-fc24e234b4ae tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.753467 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-f09fb4ca-9cc3-48c0-9bc7-c472663b71ea tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lazy-loading 'flavor' on Instance uuid 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:01:22.860717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-88f13359-3ad6-40c9-9b93-fc24e234b4ae tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:22.869581 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-88f13359-3ad6-40c9-9b93-fc24e234b4ae tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1315 microversion: 2.1 time: 0.197189 Oct 20 09:01:22.870619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 72/123] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:22 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1315 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:22.902157 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-70af2e0f-19a0-46da-a6b3-4a8031aceeb0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:22.903370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-70af2e0f-19a0-46da-a6b3-4a8031aceeb0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:22.910246 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-70af2e0f-19a0-46da-a6b3-4a8031aceeb0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.910757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-70af2e0f-19a0-46da-a6b3-4a8031aceeb0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.911236 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-70af2e0f-19a0-46da-a6b3-4a8031aceeb0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.911918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-70af2e0f-19a0-46da-a6b3-4a8031aceeb0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.912366 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-70af2e0f-19a0-46da-a6b3-4a8031aceeb0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.913339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-70af2e0f-19a0-46da-a6b3-4a8031aceeb0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.954348 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f09fb4ca-9cc3-48c0-9bc7-c472663b71ea tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7/action" status: 200 len: 32 microversion: 2.1 time: 0.302258 Oct 20 09:01:22.960067 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 52/124] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:01:22 2022] POST /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7/action => generated 32 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 20 09:01:22.966819 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5665f987-8937-4070-8d6a-07bd6da8f7dc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:22.967673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5665f987-8937-4070-8d6a-07bd6da8f7dc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:22.987468 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5665f987-8937-4070-8d6a-07bd6da8f7dc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.987954 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5665f987-8937-4070-8d6a-07bd6da8f7dc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.988450 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5665f987-8937-4070-8d6a-07bd6da8f7dc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:22.989113 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5665f987-8937-4070-8d6a-07bd6da8f7dc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:22.989565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5665f987-8937-4070-8d6a-07bd6da8f7dc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:22.990911 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5665f987-8937-4070-8d6a-07bd6da8f7dc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:23.141882 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-70af2e0f-19a0-46da-a6b3-4a8031aceeb0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:23.157369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-70af2e0f-19a0-46da-a6b3-4a8031aceeb0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.263276 Oct 20 09:01:23.158537 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 73/125] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:22 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:23.296322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5665f987-8937-4070-8d6a-07bd6da8f7dc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:23.306360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5665f987-8937-4070-8d6a-07bd6da8f7dc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.342134 Oct 20 09:01:23.307293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 53/126] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:22 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:23.325084 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-3da83e15-0a19-4ce6-b16f-abf69116bbe5 req-a21f55ca-cd24-49dc-be32-4e3f0826ea36 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f8d61907-24ca-4e74-917e-d218d0f89557", "name": "network-vif-plugged", "status": "completed", "tag": "6ba2009e-08be-4c65-a1d5-5e9548c8d407"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:23.336960 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-3da83e15-0a19-4ce6-b16f-abf69116bbe5 req-a21f55ca-cd24-49dc-be32-4e3f0826ea36 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:23.337513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-3da83e15-0a19-4ce6-b16f-abf69116bbe5 req-a21f55ca-cd24-49dc-be32-4e3f0826ea36 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:23.338014 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-3da83e15-0a19-4ce6-b16f-abf69116bbe5 req-a21f55ca-cd24-49dc-be32-4e3f0826ea36 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:23.355876 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-3da83e15-0a19-4ce6-b16f-abf69116bbe5 req-a21f55ca-cd24-49dc-be32-4e3f0826ea36 service nova] Creating event network-vif-plugged:6ba2009e-08be-4c65-a1d5-5e9548c8d407 for instance f8d61907-24ca-4e74-917e-d218d0f89557 on hv-cinder-maste Oct 20 09:01:23.373089 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-3da83e15-0a19-4ce6-b16f-abf69116bbe5 req-a21f55ca-cd24-49dc-be32-4e3f0826ea36 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041937 Oct 20 09:01:23.374343 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 74/127] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:23.414617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:23.415249 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:23.422658 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:23.423313 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:23.424053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:23.424901 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:23.425634 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:23.426260 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:23.587078 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:23.603580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:23.605514 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:23.607076 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2a60524a-9946-4fea-afe7-aae3669c74b4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.198036 Oct 20 09:01:23.608423 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 54/128] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:23 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:23.895786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-11951452-6dcf-4965-a1a5-335abe70e485 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:23.897220 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-11951452-6dcf-4965-a1a5-335abe70e485 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:23.905242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11951452-6dcf-4965-a1a5-335abe70e485 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:23.905746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11951452-6dcf-4965-a1a5-335abe70e485 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:23.906510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11951452-6dcf-4965-a1a5-335abe70e485 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:23.907502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11951452-6dcf-4965-a1a5-335abe70e485 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:23.908180 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11951452-6dcf-4965-a1a5-335abe70e485 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:23.908708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11951452-6dcf-4965-a1a5-335abe70e485 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:24.057899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-11951452-6dcf-4965-a1a5-335abe70e485 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:24.067625 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-11951452-6dcf-4965-a1a5-335abe70e485 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1502 microversion: 2.1 time: 0.177765 Oct 20 09:01:24.068068 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 75/129] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:23 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1502 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:24.179318 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-97ddcf5f-ca61-48f2-881e-da46e90826e0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:24.179927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-97ddcf5f-ca61-48f2-881e-da46e90826e0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:24.186039 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-97ddcf5f-ca61-48f2-881e-da46e90826e0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:24.186767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-97ddcf5f-ca61-48f2-881e-da46e90826e0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:24.187598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-97ddcf5f-ca61-48f2-881e-da46e90826e0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:24.188355 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-97ddcf5f-ca61-48f2-881e-da46e90826e0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:24.189221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-97ddcf5f-ca61-48f2-881e-da46e90826e0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:24.189652 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-97ddcf5f-ca61-48f2-881e-da46e90826e0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:24.334494 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-76912b60-4f53-47ab-9754-7afc96152d7f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:24.335471 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-76912b60-4f53-47ab-9754-7afc96152d7f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:24.347396 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76912b60-4f53-47ab-9754-7afc96152d7f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:24.348150 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76912b60-4f53-47ab-9754-7afc96152d7f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:24.348798 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76912b60-4f53-47ab-9754-7afc96152d7f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:24.349707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76912b60-4f53-47ab-9754-7afc96152d7f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:24.350351 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76912b60-4f53-47ab-9754-7afc96152d7f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:24.351061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76912b60-4f53-47ab-9754-7afc96152d7f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:24.459409 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-97ddcf5f-ca61-48f2-881e-da46e90826e0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:24.493520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-97ddcf5f-ca61-48f2-881e-da46e90826e0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1365 microversion: 2.1 time: 0.317750 Oct 20 09:01:24.494137 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 55/130] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:24 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1365 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:24.633004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:24.633956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:24.642178 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:24.642713 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:24.643359 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:24.644453 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:24.645417 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:24.646022 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:24.654030 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-76912b60-4f53-47ab-9754-7afc96152d7f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:24.667215 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-76912b60-4f53-47ab-9754-7afc96152d7f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.337078 Oct 20 09:01:24.671841 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 76/131] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:24 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:24.873617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:24.882608 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:24.884033 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:24.885288 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a0a669bd-7618-41d4-a44b-322f3b11f9c8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.256732 Oct 20 09:01:24.886288 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 56/132] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:24 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:25.520285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f72b7c0c-4d7e-4d64-a40a-e3f061cdb6a4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:25.520882 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f72b7c0c-4d7e-4d64-a40a-e3f061cdb6a4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:25.527964 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f72b7c0c-4d7e-4d64-a40a-e3f061cdb6a4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:25.528660 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f72b7c0c-4d7e-4d64-a40a-e3f061cdb6a4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:25.529187 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f72b7c0c-4d7e-4d64-a40a-e3f061cdb6a4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:25.530188 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f72b7c0c-4d7e-4d64-a40a-e3f061cdb6a4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:25.530845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f72b7c0c-4d7e-4d64-a40a-e3f061cdb6a4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:25.531260 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f72b7c0c-4d7e-4d64-a40a-e3f061cdb6a4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:25.714741 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-47539df7-0461-4d14-9f72-44e3b1d1626c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:25.714741 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-47539df7-0461-4d14-9f72-44e3b1d1626c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:25.728776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47539df7-0461-4d14-9f72-44e3b1d1626c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:25.729269 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47539df7-0461-4d14-9f72-44e3b1d1626c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:25.729700 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47539df7-0461-4d14-9f72-44e3b1d1626c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:25.734564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47539df7-0461-4d14-9f72-44e3b1d1626c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:25.735046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47539df7-0461-4d14-9f72-44e3b1d1626c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:25.735483 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47539df7-0461-4d14-9f72-44e3b1d1626c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:25.878094 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f72b7c0c-4d7e-4d64-a40a-e3f061cdb6a4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:25.892588 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f72b7c0c-4d7e-4d64-a40a-e3f061cdb6a4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1554 microversion: 2.1 time: 0.376051 Oct 20 09:01:25.893950 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 77/133] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:25 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1554 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:25.901348 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4a414a05-4c47-44de-ba95-688f1d6b8203 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "531f0dcb-cd16-4502-a848-12b160201023"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:25.905280 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4a414a05-4c47-44de-ba95-688f1d6b8203 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:25.914442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a414a05-4c47-44de-ba95-688f1d6b8203 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:25.915953 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a414a05-4c47-44de-ba95-688f1d6b8203 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:25.916664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a414a05-4c47-44de-ba95-688f1d6b8203 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:25.918748 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a414a05-4c47-44de-ba95-688f1d6b8203 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:25.919523 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a414a05-4c47-44de-ba95-688f1d6b8203 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:25.921423 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a414a05-4c47-44de-ba95-688f1d6b8203 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:26.100307 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-47539df7-0461-4d14-9f72-44e3b1d1626c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:26.109223 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-47539df7-0461-4d14-9f72-44e3b1d1626c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.408967 Oct 20 09:01:26.110517 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 57/134] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:25 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:26.127312 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:26.127917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:26.135000 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:26.135480 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:26.135908 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:26.136593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:26.137025 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:26.137775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:26.314250 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:26.325857 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:26.327714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:26.328902 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-06553ae0-d322-4b3d-9b82-d5a6ab79264c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.217126 Oct 20 09:01:26.330037 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 58/135] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:26 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:26.338975 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-65f8c37f-7616-4798-a9e4-c4bda4662e76 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:26.339597 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-65f8c37f-7616-4798-a9e4-c4bda4662e76 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:26.346088 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65f8c37f-7616-4798-a9e4-c4bda4662e76 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:26.346917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65f8c37f-7616-4798-a9e4-c4bda4662e76 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:26.347817 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65f8c37f-7616-4798-a9e4-c4bda4662e76 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:26.348716 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65f8c37f-7616-4798-a9e4-c4bda4662e76 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:26.349371 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65f8c37f-7616-4798-a9e4-c4bda4662e76 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:26.349977 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65f8c37f-7616-4798-a9e4-c4bda4662e76 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:26.512333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-65f8c37f-7616-4798-a9e4-c4bda4662e76 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.181003 Oct 20 09:01:26.512929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 59/136] 10.222.0.36 () {58 vars in 1275 bytes} [Thu Oct 20 09:01:26 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-interface => generated 284 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:27.135436 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b3917087-42f9-4bf8-b0b4-b65f0d143bc4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:27.136170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b3917087-42f9-4bf8-b0b4-b65f0d143bc4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:27.144805 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b3917087-42f9-4bf8-b0b4-b65f0d143bc4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:27.146346 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b3917087-42f9-4bf8-b0b4-b65f0d143bc4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:27.147553 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b3917087-42f9-4bf8-b0b4-b65f0d143bc4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:27.149204 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b3917087-42f9-4bf8-b0b4-b65f0d143bc4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:27.150211 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b3917087-42f9-4bf8-b0b4-b65f0d143bc4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:27.150326 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4a414a05-4c47-44de-ba95-688f1d6b8203 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "POST /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.254178 Oct 20 09:01:27.151247 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b3917087-42f9-4bf8-b0b4-b65f0d143bc4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:27.151601 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 78/137] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:01:25 2022] POST /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb/os-volume_attachments => generated 194 bytes in 1257 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:27.356828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:27.357517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:27.376163 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:27.377012 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:27.378817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:27.380669 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:27.381340 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:27.385788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:27.471477 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b3917087-42f9-4bf8-b0b4-b65f0d143bc4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:27.479990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b3917087-42f9-4bf8-b0b4-b65f0d143bc4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.349245 Oct 20 09:01:27.494384 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 60/138] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:27 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:27.770156 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:27.777272 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:27.778838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:27.780328 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f83226f9-8796-40c7-8b54-df2915a28acb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.427810 Oct 20 09:01:27.781114 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 79/139] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:27 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:28.298699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-905eb3ab-04c5-4c57-8e4d-eee191b464f9 req-18840c50-914a-4d3e-a077-6ff6fea42f8d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ec9ae57d-3e5a-490f-beba-d071e86a7b31", "tag": "7681f5a2-5dbc-4513-b44a-9c70525ecfee"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:28.316498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-905eb3ab-04c5-4c57-8e4d-eee191b464f9 req-18840c50-914a-4d3e-a077-6ff6fea42f8d service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:28.317973 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-905eb3ab-04c5-4c57-8e4d-eee191b464f9 req-18840c50-914a-4d3e-a077-6ff6fea42f8d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:28.321901 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-905eb3ab-04c5-4c57-8e4d-eee191b464f9 req-18840c50-914a-4d3e-a077-6ff6fea42f8d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:28.351864 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-905eb3ab-04c5-4c57-8e4d-eee191b464f9 req-18840c50-914a-4d3e-a077-6ff6fea42f8d service nova] Creating event network-changed:7681f5a2-5dbc-4513-b44a-9c70525ecfee for instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 on hv-cinder-maste Oct 20 09:01:28.403474 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-905eb3ab-04c5-4c57-8e4d-eee191b464f9 req-18840c50-914a-4d3e-a077-6ff6fea42f8d service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.091706 Oct 20 09:01:28.404348 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 61/140] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:28.504254 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f0594172-f6b3-4741-9c02-4c479a8097ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:28.505520 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f0594172-f6b3-4741-9c02-4c479a8097ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:28.517061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0594172-f6b3-4741-9c02-4c479a8097ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:28.520108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0594172-f6b3-4741-9c02-4c479a8097ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:28.521118 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0594172-f6b3-4741-9c02-4c479a8097ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:28.523729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0594172-f6b3-4741-9c02-4c479a8097ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:28.524652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0594172-f6b3-4741-9c02-4c479a8097ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:28.525292 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0594172-f6b3-4741-9c02-4c479a8097ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:28.697364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f0594172-f6b3-4741-9c02-4c479a8097ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:28.709376 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f0594172-f6b3-4741-9c02-4c479a8097ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.209586 Oct 20 09:01:28.710035 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 80/141] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:28 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:28.809054 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:28.809054 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:28.815601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:28.816379 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:28.817120 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:28.817824 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:28.818332 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:28.818777 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:28.976713 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:28.986677 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:28.988219 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:28.990338 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7db22d3a-3760-44b0-9837-aae4b6a33e9e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.188649 Oct 20 09:01:28.991018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 62/142] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:28 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:29.733014 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-43d24279-7c01-4600-94a1-9b032cceb84e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:29.733643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-43d24279-7c01-4600-94a1-9b032cceb84e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:29.760450 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-43d24279-7c01-4600-94a1-9b032cceb84e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:29.762380 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-43d24279-7c01-4600-94a1-9b032cceb84e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:29.763513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-43d24279-7c01-4600-94a1-9b032cceb84e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:29.765438 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-43d24279-7c01-4600-94a1-9b032cceb84e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:29.766109 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-43d24279-7c01-4600-94a1-9b032cceb84e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:29.767308 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-43d24279-7c01-4600-94a1-9b032cceb84e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:30.018062 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:30.018720 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:30.028866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-43d24279-7c01-4600-94a1-9b032cceb84e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:30.029923 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:30.030420 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:30.030973 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:30.031724 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:30.032205 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:30.032772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:30.040874 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-43d24279-7c01-4600-94a1-9b032cceb84e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.311428 Oct 20 09:01:30.048640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 81/143] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:29 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:30.306489 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:30.320403 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:30.323230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:30.326629 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d01b41cd-7db5-4fce-8118-93bcb0baf28b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.312792 Oct 20 09:01:30.328026 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 63/144] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:30 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:31.076332 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-89e2493c-20ca-404e-a627-c343229619d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:31.076931 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-89e2493c-20ca-404e-a627-c343229619d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:31.083204 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89e2493c-20ca-404e-a627-c343229619d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:31.083678 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89e2493c-20ca-404e-a627-c343229619d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:31.084105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89e2493c-20ca-404e-a627-c343229619d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:31.084783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89e2493c-20ca-404e-a627-c343229619d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:31.085797 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89e2493c-20ca-404e-a627-c343229619d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:31.086221 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89e2493c-20ca-404e-a627-c343229619d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:31.216622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-89e2493c-20ca-404e-a627-c343229619d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:31.224089 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-89e2493c-20ca-404e-a627-c343229619d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.151577 Oct 20 09:01:31.224649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 82/145] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:31 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:31.352520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:31.353055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:31.358922 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:31.359392 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:31.359820 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:31.360497 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:31.360936 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:31.361349 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:31.376048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:31.490994 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:31.499638 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:31.501067 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:31.502145 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b143deaf-bb5d-4187-83ea-be057222792a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.154176 Oct 20 09:01:31.502726 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 64/146] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:31 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:32.247231 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-641d1c92-94f1-4ebf-8834-52c0dd489471 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:32.248408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-641d1c92-94f1-4ebf-8834-52c0dd489471 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:32.250898 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=93 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:32.256039 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-641d1c92-94f1-4ebf-8834-52c0dd489471 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:32.256508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-641d1c92-94f1-4ebf-8834-52c0dd489471 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:32.256983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-641d1c92-94f1-4ebf-8834-52c0dd489471 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:32.257685 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-641d1c92-94f1-4ebf-8834-52c0dd489471 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:32.258124 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-641d1c92-94f1-4ebf-8834-52c0dd489471 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:32.258565 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-641d1c92-94f1-4ebf-8834-52c0dd489471 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:32.387659 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-641d1c92-94f1-4ebf-8834-52c0dd489471 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:32.395320 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-641d1c92-94f1-4ebf-8834-52c0dd489471 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.152021 Oct 20 09:01:32.395893 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 83/147] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:32 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:32.526263 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:32.526830 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:32.532710 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:32.533181 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:32.533607 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:32.534273 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:32.534742 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:32.535165 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:32.659511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:32.668120 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:32.669540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:32.670645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-af1b9765-a182-4538-a41b-9d14d30861aa tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1405 microversion: 2.20 time: 0.148758 Oct 20 09:01:32.671215 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 65/148] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:32 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1405 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:33.417052 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-38cefc2f-9876-4551-9ac1-36e33fedef9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:33.417655 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-38cefc2f-9876-4551-9ac1-36e33fedef9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:33.423985 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38cefc2f-9876-4551-9ac1-36e33fedef9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:33.424459 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38cefc2f-9876-4551-9ac1-36e33fedef9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:33.424891 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38cefc2f-9876-4551-9ac1-36e33fedef9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:33.425571 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38cefc2f-9876-4551-9ac1-36e33fedef9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:33.426022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38cefc2f-9876-4551-9ac1-36e33fedef9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:33.426442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38cefc2f-9876-4551-9ac1-36e33fedef9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:33.558790 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-38cefc2f-9876-4551-9ac1-36e33fedef9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:33.566993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-38cefc2f-9876-4551-9ac1-36e33fedef9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.153773 Oct 20 09:01:33.567715 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 84/149] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:33 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:33.694861 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:33.695844 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:33.703892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:33.704517 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:33.704980 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:33.705721 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:33.707146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:33.708176 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:33.855094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:33.865047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:33.866508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:33.868239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-65fe2100-717d-4c6a-833e-9cb0c90ce99a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1596 microversion: 2.20 time: 0.176997 Oct 20 09:01:33.868798 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 66/150] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:01:33 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1596 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:01:33.894724 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8bb42e85-e56a-4590-85f2-26ac05f8ec6f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:33.895633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8bb42e85-e56a-4590-85f2-26ac05f8ec6f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:33.902373 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8bb42e85-e56a-4590-85f2-26ac05f8ec6f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:33.903010 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8bb42e85-e56a-4590-85f2-26ac05f8ec6f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:33.903521 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8bb42e85-e56a-4590-85f2-26ac05f8ec6f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:33.904291 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8bb42e85-e56a-4590-85f2-26ac05f8ec6f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:33.904810 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8bb42e85-e56a-4590-85f2-26ac05f8ec6f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:33.905407 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8bb42e85-e56a-4590-85f2-26ac05f8ec6f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:33.989689 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8bb42e85-e56a-4590-85f2-26ac05f8ec6f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.099873 Oct 20 09:01:33.990739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 85/151] 10.222.0.36 () {60 vars in 1316 bytes} [Thu Oct 20 09:01:33 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-interface => generated 284 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 09:01:34.591521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bb38796e-ed28-463d-927f-35f3d5b1dc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:34.592126 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bb38796e-ed28-463d-927f-35f3d5b1dc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:34.598055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb38796e-ed28-463d-927f-35f3d5b1dc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:34.598565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb38796e-ed28-463d-927f-35f3d5b1dc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:34.599015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb38796e-ed28-463d-927f-35f3d5b1dc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:34.599682 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb38796e-ed28-463d-927f-35f3d5b1dc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:34.600124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb38796e-ed28-463d-927f-35f3d5b1dc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:34.600542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb38796e-ed28-463d-927f-35f3d5b1dc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:35.147887 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bb38796e-ed28-463d-927f-35f3d5b1dc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:35.157081 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bb38796e-ed28-463d-927f-35f3d5b1dc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.569619 Oct 20 09:01:35.157961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 67/152] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:34 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:35.269242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1520779657", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "networks": [{"uuid": "4ee4717f-cbab-45d5-be89-db298134faad"}]}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:35.325468 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] validate_networks() for [('4ee4717f-cbab-45d5-be89-db298134faad', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:01:35.627353 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-192c8ab6-1bee-44ef-80dc-d17718c2b035 req-0eaf7552-5f19-41c4-a989-00a8736aa078 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f8d61907-24ca-4e74-917e-d218d0f89557", "tag": "6ba2009e-08be-4c65-a1d5-5e9548c8d407"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:35.637611 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-192c8ab6-1bee-44ef-80dc-d17718c2b035 req-0eaf7552-5f19-41c4-a989-00a8736aa078 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:35.638091 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-192c8ab6-1bee-44ef-80dc-d17718c2b035 req-0eaf7552-5f19-41c4-a989-00a8736aa078 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:35.638541 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-192c8ab6-1bee-44ef-80dc-d17718c2b035 req-0eaf7552-5f19-41c4-a989-00a8736aa078 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:35.651083 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-192c8ab6-1bee-44ef-80dc-d17718c2b035 req-0eaf7552-5f19-41c4-a989-00a8736aa078 service nova] Creating event network-changed:6ba2009e-08be-4c65-a1d5-5e9548c8d407 for instance f8d61907-24ca-4e74-917e-d218d0f89557 on hv-cinder-maste Oct 20 09:01:35.657433 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-192c8ab6-1bee-44ef-80dc-d17718c2b035 req-0eaf7552-5f19-41c4-a989-00a8736aa078 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033577 Oct 20 09:01:35.658669 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 68/153] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:35.713110 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:01:35.713529 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:01:35.714061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:01:35.714494 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:01:35.714904 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:01:35.724628 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:35.725406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:35.726097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:35.726998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:35.727681 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:35.728128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:35.745627 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Getting quotas for project 847f0b8423334fc7b505313a10845bcb. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:01:35.749711 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Getting quotas for user 930afcdb977f4ed2a33b366563a755f1 and project 847f0b8423334fc7b505313a10845bcb. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:01:35.761845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:01:35.764265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:01:35.810022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-05d30d36-4e97-427a-8104-9a517f8af348 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.544977 Oct 20 09:01:35.810690 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 86/154] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:01:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 546 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:01:35.823272 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-48a2a51f-1356-4bf2-bce1-13b12acbb324 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:35.823958 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-48a2a51f-1356-4bf2-bce1-13b12acbb324 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:35.881513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-48a2a51f-1356-4bf2-bce1-13b12acbb324 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:35.890237 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-48a2a51f-1356-4bf2-bce1-13b12acbb324 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1214 microversion: 2.1 time: 0.071780 Oct 20 09:01:35.890834 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 69/155] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:35 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1214 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:36.191970 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ce90a09d-6cf5-4fda-a433-639565d1ed00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:36.192804 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ce90a09d-6cf5-4fda-a433-639565d1ed00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:36.209980 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce90a09d-6cf5-4fda-a433-639565d1ed00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:36.210501 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce90a09d-6cf5-4fda-a433-639565d1ed00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:36.211657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce90a09d-6cf5-4fda-a433-639565d1ed00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:36.212433 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce90a09d-6cf5-4fda-a433-639565d1ed00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:36.212876 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce90a09d-6cf5-4fda-a433-639565d1ed00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:36.213338 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce90a09d-6cf5-4fda-a433-639565d1ed00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:36.424437 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ce90a09d-6cf5-4fda-a433-639565d1ed00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:36.433982 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ce90a09d-6cf5-4fda-a433-639565d1ed00 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.257726 Oct 20 09:01:36.435161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 87/156] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:36 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:36.914555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-36d90097-5030-4389-be49-f1f8f336b4d8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:36.915203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-36d90097-5030-4389-be49-f1f8f336b4d8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:36.932230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36d90097-5030-4389-be49-f1f8f336b4d8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:36.932230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36d90097-5030-4389-be49-f1f8f336b4d8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:36.932230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36d90097-5030-4389-be49-f1f8f336b4d8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:36.932230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36d90097-5030-4389-be49-f1f8f336b4d8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:36.934273 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36d90097-5030-4389-be49-f1f8f336b4d8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:36.934805 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36d90097-5030-4389-be49-f1f8f336b4d8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:37.088659 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-36d90097-5030-4389-be49-f1f8f336b4d8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:37.102361 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-36d90097-5030-4389-be49-f1f8f336b4d8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1266 microversion: 2.1 time: 0.192391 Oct 20 09:01:37.102984 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 70/157] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:36 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1266 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:37.458062 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f037c5a5-f07d-4526-ba24-9515d7082f20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:37.458791 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f037c5a5-f07d-4526-ba24-9515d7082f20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:37.465515 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f037c5a5-f07d-4526-ba24-9515d7082f20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:37.466070 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f037c5a5-f07d-4526-ba24-9515d7082f20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:37.466597 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f037c5a5-f07d-4526-ba24-9515d7082f20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:37.467491 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f037c5a5-f07d-4526-ba24-9515d7082f20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:37.468153 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f037c5a5-f07d-4526-ba24-9515d7082f20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:37.468638 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f037c5a5-f07d-4526-ba24-9515d7082f20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:37.615802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f037c5a5-f07d-4526-ba24-9515d7082f20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:37.620280 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:37.625168 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f037c5a5-f07d-4526-ba24-9515d7082f20 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.171178 Oct 20 09:01:37.625731 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 88/158] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:37 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:38.126147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-47643fb3-422f-49fc-a40c-a130e2489a71 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:38.126950 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-47643fb3-422f-49fc-a40c-a130e2489a71 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:38.142792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=84 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:38.142792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47643fb3-422f-49fc-a40c-a130e2489a71 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:38.142792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47643fb3-422f-49fc-a40c-a130e2489a71 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:38.142792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47643fb3-422f-49fc-a40c-a130e2489a71 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:38.143069 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47643fb3-422f-49fc-a40c-a130e2489a71 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:38.143069 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47643fb3-422f-49fc-a40c-a130e2489a71 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:38.143069 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-47643fb3-422f-49fc-a40c-a130e2489a71 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:38.295844 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-47643fb3-422f-49fc-a40c-a130e2489a71 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:38.309684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-47643fb3-422f-49fc-a40c-a130e2489a71 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1272 microversion: 2.1 time: 0.188121 Oct 20 09:01:38.320052 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 71/159] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:38 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1272 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:38.647771 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-91942f64-8301-4a5c-b41a-eff777ad51b1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:38.648409 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-91942f64-8301-4a5c-b41a-eff777ad51b1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:38.655594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91942f64-8301-4a5c-b41a-eff777ad51b1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:38.656281 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91942f64-8301-4a5c-b41a-eff777ad51b1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:38.656713 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91942f64-8301-4a5c-b41a-eff777ad51b1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:38.657400 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91942f64-8301-4a5c-b41a-eff777ad51b1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:38.658004 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91942f64-8301-4a5c-b41a-eff777ad51b1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:38.658427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91942f64-8301-4a5c-b41a-eff777ad51b1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:38.853481 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-91942f64-8301-4a5c-b41a-eff777ad51b1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:38.861584 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-91942f64-8301-4a5c-b41a-eff777ad51b1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.218263 Oct 20 09:01:38.862346 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 89/160] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:38 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:39.334293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3f46bbe7-f58d-4bb0-9ecc-c09388f247fd tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:39.334971 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3f46bbe7-f58d-4bb0-9ecc-c09388f247fd tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:39.341937 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3f46bbe7-f58d-4bb0-9ecc-c09388f247fd tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:39.342587 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3f46bbe7-f58d-4bb0-9ecc-c09388f247fd tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:39.343021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3f46bbe7-f58d-4bb0-9ecc-c09388f247fd tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:39.343709 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3f46bbe7-f58d-4bb0-9ecc-c09388f247fd tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:39.344156 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3f46bbe7-f58d-4bb0-9ecc-c09388f247fd tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:39.344580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3f46bbe7-f58d-4bb0-9ecc-c09388f247fd tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:39.481573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3f46bbe7-f58d-4bb0-9ecc-c09388f247fd tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:39.489188 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3f46bbe7-f58d-4bb0-9ecc-c09388f247fd tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.158840 Oct 20 09:01:39.489745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 72/161] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:39 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:39.885779 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-289f3bcb-d08d-4762-9596-371189856ee4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:39.886389 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-289f3bcb-d08d-4762-9596-371189856ee4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:39.894633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-289f3bcb-d08d-4762-9596-371189856ee4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:39.895116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-289f3bcb-d08d-4762-9596-371189856ee4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:39.895704 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-289f3bcb-d08d-4762-9596-371189856ee4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:39.896537 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-289f3bcb-d08d-4762-9596-371189856ee4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:39.897242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-289f3bcb-d08d-4762-9596-371189856ee4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:39.897696 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-289f3bcb-d08d-4762-9596-371189856ee4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:40.045272 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-289f3bcb-d08d-4762-9596-371189856ee4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:40.053412 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-289f3bcb-d08d-4762-9596-371189856ee4 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.172668 Oct 20 09:01:40.054245 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 90/162] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:39 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:40.512619 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ca30a838-7926-48e3-ac90-dc7134014156 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:40.513407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ca30a838-7926-48e3-ac90-dc7134014156 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:40.519285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca30a838-7926-48e3-ac90-dc7134014156 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:40.519759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca30a838-7926-48e3-ac90-dc7134014156 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:40.520185 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca30a838-7926-48e3-ac90-dc7134014156 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:40.521122 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca30a838-7926-48e3-ac90-dc7134014156 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:40.521566 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca30a838-7926-48e3-ac90-dc7134014156 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:40.522018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca30a838-7926-48e3-ac90-dc7134014156 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:40.662818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ca30a838-7926-48e3-ac90-dc7134014156 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:40.670812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ca30a838-7926-48e3-ac90-dc7134014156 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.161834 Oct 20 09:01:40.671377 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 73/163] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:40 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:41.075936 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-94cad2e7-47d4-40d2-b1fd-e297a97478b8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:41.076539 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-94cad2e7-47d4-40d2-b1fd-e297a97478b8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:41.082707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-94cad2e7-47d4-40d2-b1fd-e297a97478b8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:41.083187 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-94cad2e7-47d4-40d2-b1fd-e297a97478b8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:41.083620 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-94cad2e7-47d4-40d2-b1fd-e297a97478b8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:41.084310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-94cad2e7-47d4-40d2-b1fd-e297a97478b8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:41.084757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-94cad2e7-47d4-40d2-b1fd-e297a97478b8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:41.085177 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-94cad2e7-47d4-40d2-b1fd-e297a97478b8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:41.249951 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-94cad2e7-47d4-40d2-b1fd-e297a97478b8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:41.258457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-94cad2e7-47d4-40d2-b1fd-e297a97478b8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.185721 Oct 20 09:01:41.259160 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 91/164] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:41 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:41.693418 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4cc97ef9-d055-41de-a602-5d8402588d3a tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:41.694030 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4cc97ef9-d055-41de-a602-5d8402588d3a tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:41.700560 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4cc97ef9-d055-41de-a602-5d8402588d3a tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:41.701043 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4cc97ef9-d055-41de-a602-5d8402588d3a tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:41.701518 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4cc97ef9-d055-41de-a602-5d8402588d3a tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:41.702150 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4cc97ef9-d055-41de-a602-5d8402588d3a tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:41.702625 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4cc97ef9-d055-41de-a602-5d8402588d3a tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:41.703047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4cc97ef9-d055-41de-a602-5d8402588d3a tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:41.833484 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4cc97ef9-d055-41de-a602-5d8402588d3a tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:41.843071 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4cc97ef9-d055-41de-a602-5d8402588d3a tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.153398 Oct 20 09:01:41.855040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 74/165] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:41 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:42.281598 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ed196790-a0d5-4336-bc94-c2b9768c7ae3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:42.282205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ed196790-a0d5-4336-bc94-c2b9768c7ae3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:42.288566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ed196790-a0d5-4336-bc94-c2b9768c7ae3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:42.289133 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ed196790-a0d5-4336-bc94-c2b9768c7ae3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:42.289563 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ed196790-a0d5-4336-bc94-c2b9768c7ae3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:42.290245 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ed196790-a0d5-4336-bc94-c2b9768c7ae3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:42.290886 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ed196790-a0d5-4336-bc94-c2b9768c7ae3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:42.291525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ed196790-a0d5-4336-bc94-c2b9768c7ae3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:42.507175 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ed196790-a0d5-4336-bc94-c2b9768c7ae3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:42.516676 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ed196790-a0d5-4336-bc94-c2b9768c7ae3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.239207 Oct 20 09:01:42.517422 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 92/166] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:42 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:42.664612 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-33e563a2-9bbd-492b-bd61-86fa16dff7c4 req-1ef28fa2-6b04-4631-bc46-f15c53f717ed service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8e67be6e-328c-4a4f-8a2b-083b26c3d62a", "tag": "a052a7b6-dd87-4142-8f12-b3b7205a2cef"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:42.673771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-33e563a2-9bbd-492b-bd61-86fa16dff7c4 req-1ef28fa2-6b04-4631-bc46-f15c53f717ed service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:42.674248 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-33e563a2-9bbd-492b-bd61-86fa16dff7c4 req-1ef28fa2-6b04-4631-bc46-f15c53f717ed service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:42.674690 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-33e563a2-9bbd-492b-bd61-86fa16dff7c4 req-1ef28fa2-6b04-4631-bc46-f15c53f717ed service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:42.688142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-33e563a2-9bbd-492b-bd61-86fa16dff7c4 req-1ef28fa2-6b04-4631-bc46-f15c53f717ed service nova] Creating event network-changed:a052a7b6-dd87-4142-8f12-b3b7205a2cef for instance 8e67be6e-328c-4a4f-8a2b-083b26c3d62a on hv-cinder-maste Oct 20 09:01:42.694315 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-33e563a2-9bbd-492b-bd61-86fa16dff7c4 req-1ef28fa2-6b04-4631-bc46-f15c53f717ed service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034042 Oct 20 09:01:42.697073 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 75/167] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:01:42.868897 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-91e980e3-6595-4e1a-945b-212c4ca4c034 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:42.870043 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-91e980e3-6595-4e1a-945b-212c4ca4c034 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:42.876595 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91e980e3-6595-4e1a-945b-212c4ca4c034 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:42.877149 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91e980e3-6595-4e1a-945b-212c4ca4c034 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:42.877642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91e980e3-6595-4e1a-945b-212c4ca4c034 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:42.878389 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91e980e3-6595-4e1a-945b-212c4ca4c034 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:42.878923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91e980e3-6595-4e1a-945b-212c4ca4c034 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:42.879600 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-91e980e3-6595-4e1a-945b-212c4ca4c034 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:43.070824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-91e980e3-6595-4e1a-945b-212c4ca4c034 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:43.078622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-91e980e3-6595-4e1a-945b-212c4ca4c034 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.214236 Oct 20 09:01:43.079196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 93/168] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:42 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:43.540564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-48fe8f3c-dcbd-4e9d-801b-33a89ab4c201 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:43.541876 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-48fe8f3c-dcbd-4e9d-801b-33a89ab4c201 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:43.548511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-48fe8f3c-dcbd-4e9d-801b-33a89ab4c201 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:43.548990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-48fe8f3c-dcbd-4e9d-801b-33a89ab4c201 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:43.549614 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-48fe8f3c-dcbd-4e9d-801b-33a89ab4c201 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:43.550310 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-48fe8f3c-dcbd-4e9d-801b-33a89ab4c201 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:43.550788 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-48fe8f3c-dcbd-4e9d-801b-33a89ab4c201 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:43.551371 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-48fe8f3c-dcbd-4e9d-801b-33a89ab4c201 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:43.702348 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-48fe8f3c-dcbd-4e9d-801b-33a89ab4c201 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:43.710565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-48fe8f3c-dcbd-4e9d-801b-33a89ab4c201 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.174304 Oct 20 09:01:43.722564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 76/169] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:43 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:44.106378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f1018363-ab62-4fbb-a401-7c331b72c34e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:44.107099 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f1018363-ab62-4fbb-a401-7c331b72c34e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:44.115396 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f1018363-ab62-4fbb-a401-7c331b72c34e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:44.115918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f1018363-ab62-4fbb-a401-7c331b72c34e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:44.116443 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f1018363-ab62-4fbb-a401-7c331b72c34e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:44.117955 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f1018363-ab62-4fbb-a401-7c331b72c34e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:44.118572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f1018363-ab62-4fbb-a401-7c331b72c34e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:44.119085 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f1018363-ab62-4fbb-a401-7c331b72c34e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:44.278809 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f1018363-ab62-4fbb-a401-7c331b72c34e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:44.286735 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f1018363-ab62-4fbb-a401-7c331b72c34e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.186013 Oct 20 09:01:44.287306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 94/170] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:44 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:44.736689 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0738b18c-490f-4e9f-80c3-1386eddf6c23 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:44.737404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0738b18c-490f-4e9f-80c3-1386eddf6c23 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:44.760466 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0738b18c-490f-4e9f-80c3-1386eddf6c23 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:44.761132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0738b18c-490f-4e9f-80c3-1386eddf6c23 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:44.761591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0738b18c-490f-4e9f-80c3-1386eddf6c23 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:44.762877 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0738b18c-490f-4e9f-80c3-1386eddf6c23 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:44.765340 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0738b18c-490f-4e9f-80c3-1386eddf6c23 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:44.765792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0738b18c-490f-4e9f-80c3-1386eddf6c23 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:44.963978 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0738b18c-490f-4e9f-80c3-1386eddf6c23 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:44.971885 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0738b18c-490f-4e9f-80c3-1386eddf6c23 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.239094 Oct 20 09:01:44.972444 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 77/171] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:44 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:45.312437 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-42365903-6729-4abd-8fe5-72a9622f1a09 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:45.313391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-42365903-6729-4abd-8fe5-72a9622f1a09 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:45.321801 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42365903-6729-4abd-8fe5-72a9622f1a09 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:45.322928 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42365903-6729-4abd-8fe5-72a9622f1a09 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:45.323493 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42365903-6729-4abd-8fe5-72a9622f1a09 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:45.324335 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42365903-6729-4abd-8fe5-72a9622f1a09 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:45.324861 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42365903-6729-4abd-8fe5-72a9622f1a09 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:45.325486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42365903-6729-4abd-8fe5-72a9622f1a09 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:45.552308 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-42365903-6729-4abd-8fe5-72a9622f1a09 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:45.568577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-42365903-6729-4abd-8fe5-72a9622f1a09 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.257056 Oct 20 09:01:45.571153 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 95/172] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:45 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:45.800925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d55f9748-ce4e-4d78-a66a-a28d0dabe1e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:45.801422 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d55f9748-ce4e-4d78-a66a-a28d0dabe1e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:45.807488 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55f9748-ce4e-4d78-a66a-a28d0dabe1e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:45.807980 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55f9748-ce4e-4d78-a66a-a28d0dabe1e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:45.808451 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55f9748-ce4e-4d78-a66a-a28d0dabe1e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:45.809339 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55f9748-ce4e-4d78-a66a-a28d0dabe1e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:45.810100 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55f9748-ce4e-4d78-a66a-a28d0dabe1e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:45.810555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55f9748-ce4e-4d78-a66a-a28d0dabe1e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:45.839163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d55f9748-ce4e-4d78-a66a-a28d0dabe1e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:01:45.890217 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:45.936800 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d55f9748-ce4e-4d78-a66a-a28d0dabe1e8 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "DELETE /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 204 len: 0 microversion: 2.1 time: 0.139580 Oct 20 09:01:45.937391 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 78/173] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:01:45 2022] DELETE /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:01:45.946521 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1c436bdf-27c4-47ae-85e7-bdca469c4a84 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:45.947801 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1c436bdf-27c4-47ae-85e7-bdca469c4a84 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:45.949637 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:45.950799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=6 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:45.958000 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1c436bdf-27c4-47ae-85e7-bdca469c4a84 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:45.958745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1c436bdf-27c4-47ae-85e7-bdca469c4a84 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:45.959853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1c436bdf-27c4-47ae-85e7-bdca469c4a84 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:45.961302 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1c436bdf-27c4-47ae-85e7-bdca469c4a84 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:45.961923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1c436bdf-27c4-47ae-85e7-bdca469c4a84 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:45.962357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1c436bdf-27c4-47ae-85e7-bdca469c4a84 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:45.995170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bd104e0f-86ae-4f6a-a7cd-8185642ce01a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:45.995985 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bd104e0f-86ae-4f6a-a7cd-8185642ce01a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:46.001805 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd104e0f-86ae-4f6a-a7cd-8185642ce01a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:46.002287 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd104e0f-86ae-4f6a-a7cd-8185642ce01a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:46.002758 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd104e0f-86ae-4f6a-a7cd-8185642ce01a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:46.003499 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd104e0f-86ae-4f6a-a7cd-8185642ce01a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:46.004109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd104e0f-86ae-4f6a-a7cd-8185642ce01a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:46.004349 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd104e0f-86ae-4f6a-a7cd-8185642ce01a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:46.104593 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1c436bdf-27c4-47ae-85e7-bdca469c4a84 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:46.112833 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1c436bdf-27c4-47ae-85e7-bdca469c4a84 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1554 microversion: 2.1 time: 0.170656 Oct 20 09:01:46.113401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 96/174] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:45 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1554 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:46.139536 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bd104e0f-86ae-4f6a-a7cd-8185642ce01a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:46.149471 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bd104e0f-86ae-4f6a-a7cd-8185642ce01a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.158553 Oct 20 09:01:46.150264 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 79/175] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:45 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:46.601059 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0c0b75bf-a36e-4536-ac68-608aac932062 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:46.601665 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0c0b75bf-a36e-4536-ac68-608aac932062 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:46.608261 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c0b75bf-a36e-4536-ac68-608aac932062 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:46.608746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c0b75bf-a36e-4536-ac68-608aac932062 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:46.609172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c0b75bf-a36e-4536-ac68-608aac932062 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:46.609842 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c0b75bf-a36e-4536-ac68-608aac932062 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:46.610284 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c0b75bf-a36e-4536-ac68-608aac932062 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:46.610732 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c0b75bf-a36e-4536-ac68-608aac932062 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:46.744941 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0c0b75bf-a36e-4536-ac68-608aac932062 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:46.753381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0c0b75bf-a36e-4536-ac68-608aac932062 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.156060 Oct 20 09:01:46.766815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 97/176] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:46 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:47.136272 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-701d0d72-c2de-48e5-9d95-5f26cf28fa0c tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:47.136802 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-701d0d72-c2de-48e5-9d95-5f26cf28fa0c tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:47.142939 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-701d0d72-c2de-48e5-9d95-5f26cf28fa0c tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:47.143774 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-701d0d72-c2de-48e5-9d95-5f26cf28fa0c tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:47.143993 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-701d0d72-c2de-48e5-9d95-5f26cf28fa0c tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:47.144625 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-701d0d72-c2de-48e5-9d95-5f26cf28fa0c tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:47.145067 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-701d0d72-c2de-48e5-9d95-5f26cf28fa0c tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:47.145506 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-701d0d72-c2de-48e5-9d95-5f26cf28fa0c tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:47.180715 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-371ee8be-72f5-4e7d-895e-bb34871e8a7b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:47.180715 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-371ee8be-72f5-4e7d-895e-bb34871e8a7b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:47.189003 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-371ee8be-72f5-4e7d-895e-bb34871e8a7b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:47.189495 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-371ee8be-72f5-4e7d-895e-bb34871e8a7b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:47.189932 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-371ee8be-72f5-4e7d-895e-bb34871e8a7b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:47.190669 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-371ee8be-72f5-4e7d-895e-bb34871e8a7b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:47.191118 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-371ee8be-72f5-4e7d-895e-bb34871e8a7b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:47.191544 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-371ee8be-72f5-4e7d-895e-bb34871e8a7b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:47.275515 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-701d0d72-c2de-48e5-9d95-5f26cf28fa0c tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:47.283757 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-701d0d72-c2de-48e5-9d95-5f26cf28fa0c tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1554 microversion: 2.1 time: 0.151819 Oct 20 09:01:47.284344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 80/177] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:47 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1554 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:47.322448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-371ee8be-72f5-4e7d-895e-bb34871e8a7b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:47.330612 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-371ee8be-72f5-4e7d-895e-bb34871e8a7b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.161478 Oct 20 09:01:47.331176 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 98/178] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:47 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:47.776570 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9ea7a50a-dcc7-4d8a-8056-91c0cc5fc5e8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:47.777105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-9ea7a50a-dcc7-4d8a-8056-91c0cc5fc5e8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:47.783228 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ea7a50a-dcc7-4d8a-8056-91c0cc5fc5e8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:47.783709 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ea7a50a-dcc7-4d8a-8056-91c0cc5fc5e8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:47.784142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ea7a50a-dcc7-4d8a-8056-91c0cc5fc5e8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:47.784817 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ea7a50a-dcc7-4d8a-8056-91c0cc5fc5e8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:47.785259 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ea7a50a-dcc7-4d8a-8056-91c0cc5fc5e8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:47.785676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ea7a50a-dcc7-4d8a-8056-91c0cc5fc5e8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:47.920859 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-9ea7a50a-dcc7-4d8a-8056-91c0cc5fc5e8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:47.923156 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:47.930409 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-9ea7a50a-dcc7-4d8a-8056-91c0cc5fc5e8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.157732 Oct 20 09:01:47.931000 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 81/179] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:47 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:48.312478 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e4c67c76-3c3d-465f-af97-331f1942c5c5 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:48.313241 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-e4c67c76-3c3d-465f-af97-331f1942c5c5 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:48.319455 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e4c67c76-3c3d-465f-af97-331f1942c5c5 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:48.320352 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e4c67c76-3c3d-465f-af97-331f1942c5c5 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:48.321284 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e4c67c76-3c3d-465f-af97-331f1942c5c5 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:48.322424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e4c67c76-3c3d-465f-af97-331f1942c5c5 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:48.323484 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e4c67c76-3c3d-465f-af97-331f1942c5c5 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:48.324119 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e4c67c76-3c3d-465f-af97-331f1942c5c5 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:48.357212 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ae3ce587-2a2c-411e-8eb6-28296ee75161 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:48.358717 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ae3ce587-2a2c-411e-8eb6-28296ee75161 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:48.375319 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae3ce587-2a2c-411e-8eb6-28296ee75161 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:48.376311 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae3ce587-2a2c-411e-8eb6-28296ee75161 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:48.376919 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae3ce587-2a2c-411e-8eb6-28296ee75161 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:48.379772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae3ce587-2a2c-411e-8eb6-28296ee75161 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:48.380668 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae3ce587-2a2c-411e-8eb6-28296ee75161 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:48.381306 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae3ce587-2a2c-411e-8eb6-28296ee75161 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:48.500281 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e4c67c76-3c3d-465f-af97-331f1942c5c5 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:48.516570 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e4c67c76-3c3d-465f-af97-331f1942c5c5 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1554 microversion: 2.1 time: 0.211795 Oct 20 09:01:48.517594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 99/180] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:48 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1554 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:48.565108 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ae3ce587-2a2c-411e-8eb6-28296ee75161 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:48.573759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ae3ce587-2a2c-411e-8eb6-28296ee75161 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.224185 Oct 20 09:01:48.586542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 82/181] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:48 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:48.953274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-27c3a440-d799-43f5-942d-93aa2d542de4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:48.953924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-27c3a440-d799-43f5-942d-93aa2d542de4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:48.959956 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c3a440-d799-43f5-942d-93aa2d542de4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:48.960951 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c3a440-d799-43f5-942d-93aa2d542de4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:48.961487 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c3a440-d799-43f5-942d-93aa2d542de4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:48.962171 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c3a440-d799-43f5-942d-93aa2d542de4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:48.962664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c3a440-d799-43f5-942d-93aa2d542de4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:48.963083 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c3a440-d799-43f5-942d-93aa2d542de4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:49.107890 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-27c3a440-d799-43f5-942d-93aa2d542de4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:49.116092 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-27c3a440-d799-43f5-942d-93aa2d542de4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.166900 Oct 20 09:01:49.116755 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 100/182] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:48 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:49.541914 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c66d6f99-021c-4de7-814f-4a198898f25d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:49.543367 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c66d6f99-021c-4de7-814f-4a198898f25d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:49.550150 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c66d6f99-021c-4de7-814f-4a198898f25d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:49.551340 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c66d6f99-021c-4de7-814f-4a198898f25d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:49.551992 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c66d6f99-021c-4de7-814f-4a198898f25d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:49.552993 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c66d6f99-021c-4de7-814f-4a198898f25d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:49.553607 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c66d6f99-021c-4de7-814f-4a198898f25d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:49.554346 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c66d6f99-021c-4de7-814f-4a198898f25d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:49.598100 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d2d3adb6-f9b7-48f9-bd4f-ffda781e2c58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:49.599341 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d2d3adb6-f9b7-48f9-bd4f-ffda781e2c58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:49.608105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2d3adb6-f9b7-48f9-bd4f-ffda781e2c58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:49.608813 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2d3adb6-f9b7-48f9-bd4f-ffda781e2c58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:49.609462 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2d3adb6-f9b7-48f9-bd4f-ffda781e2c58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:49.610842 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2d3adb6-f9b7-48f9-bd4f-ffda781e2c58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:49.611496 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2d3adb6-f9b7-48f9-bd4f-ffda781e2c58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:49.612518 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2d3adb6-f9b7-48f9-bd4f-ffda781e2c58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:49.838276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d2d3adb6-f9b7-48f9-bd4f-ffda781e2c58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:49.848295 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d2d3adb6-f9b7-48f9-bd4f-ffda781e2c58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.253047 Oct 20 09:01:49.856336 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 101/183] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:49 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:50.091996 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c66d6f99-021c-4de7-814f-4a198898f25d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:50.108804 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c66d6f99-021c-4de7-814f-4a198898f25d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1554 microversion: 2.1 time: 0.570447 Oct 20 09:01:50.111475 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 83/184] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:49 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1554 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:50.147067 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-75fc5fb0-1613-42cc-812a-ed3094728f34 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:50.147067 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-75fc5fb0-1613-42cc-812a-ed3094728f34 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:50.159952 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75fc5fb0-1613-42cc-812a-ed3094728f34 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:50.160420 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75fc5fb0-1613-42cc-812a-ed3094728f34 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:50.160883 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75fc5fb0-1613-42cc-812a-ed3094728f34 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:50.161885 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75fc5fb0-1613-42cc-812a-ed3094728f34 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:50.162340 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75fc5fb0-1613-42cc-812a-ed3094728f34 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:50.162843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75fc5fb0-1613-42cc-812a-ed3094728f34 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:50.330419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-75fc5fb0-1613-42cc-812a-ed3094728f34 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:50.339826 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-75fc5fb0-1613-42cc-812a-ed3094728f34 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.203563 Oct 20 09:01:50.341062 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 102/185] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:50 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:50.509886 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-9bcaa855-6d92-4aa9-a97a-e1e175273ece req-28406c70-bbb7-4ad0-bd95-41faa59b2efd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91bccd9d-c1c7-477f-b8a5-914edd9fa2eb", "name": "network-vif-deleted", "tag": "b5692122-4272-4400-bc7a-9069ae5bc48a"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:50.520482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-9bcaa855-6d92-4aa9-a97a-e1e175273ece req-28406c70-bbb7-4ad0-bd95-41faa59b2efd service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:50.520968 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-9bcaa855-6d92-4aa9-a97a-e1e175273ece req-28406c70-bbb7-4ad0-bd95-41faa59b2efd service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:50.521392 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-9bcaa855-6d92-4aa9-a97a-e1e175273ece req-28406c70-bbb7-4ad0-bd95-41faa59b2efd service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:50.534794 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-9bcaa855-6d92-4aa9-a97a-e1e175273ece req-28406c70-bbb7-4ad0-bd95-41faa59b2efd service nova] Creating event network-vif-deleted:b5692122-4272-4400-bc7a-9069ae5bc48a for instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb on hv-cinder-maste Oct 20 09:01:50.540412 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-9bcaa855-6d92-4aa9-a97a-e1e175273ece req-28406c70-bbb7-4ad0-bd95-41faa59b2efd service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035141 Oct 20 09:01:50.541077 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 84/186] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:01:50.872226 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3850e7be-45eb-48f5-aded-9aec45a34ea8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:50.872880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3850e7be-45eb-48f5-aded-9aec45a34ea8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:50.880441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3850e7be-45eb-48f5-aded-9aec45a34ea8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:50.880918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3850e7be-45eb-48f5-aded-9aec45a34ea8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:50.881345 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3850e7be-45eb-48f5-aded-9aec45a34ea8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:50.882023 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3850e7be-45eb-48f5-aded-9aec45a34ea8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:50.882612 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3850e7be-45eb-48f5-aded-9aec45a34ea8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:50.883029 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3850e7be-45eb-48f5-aded-9aec45a34ea8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:51.048424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3850e7be-45eb-48f5-aded-9aec45a34ea8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:51.057279 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3850e7be-45eb-48f5-aded-9aec45a34ea8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.188854 Oct 20 09:01:51.057915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 103/187] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:50 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:51.136150 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:51.136925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:51.144960 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:51.145589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:51.146153 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:51.147257 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:51.147941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:51.148971 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:51.187868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:51.188573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:51.189017 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:51.189468 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lazy-loading 'fault' on Instance uuid 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:01:51.243013 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7c40a4592874732871eaabd04d8662c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4da29c9977ca44baa867b3c7fc86164b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:51.255099 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-46b186c8-6e08-463e-aa2a-e5fb27dbaa8f tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 200 len: 1351 microversion: 2.1 time: 0.124084 Oct 20 09:01:51.270892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 85/188] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:51 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 1351 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:51.371757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-63296c33-e93b-4cfa-8174-f8a1ba817f1c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:51.372490 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-63296c33-e93b-4cfa-8174-f8a1ba817f1c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:51.378496 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63296c33-e93b-4cfa-8174-f8a1ba817f1c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:51.378994 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63296c33-e93b-4cfa-8174-f8a1ba817f1c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:51.379480 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63296c33-e93b-4cfa-8174-f8a1ba817f1c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:51.380096 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63296c33-e93b-4cfa-8174-f8a1ba817f1c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:51.380536 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63296c33-e93b-4cfa-8174-f8a1ba817f1c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:51.380945 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63296c33-e93b-4cfa-8174-f8a1ba817f1c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:51.539716 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-63296c33-e93b-4cfa-8174-f8a1ba817f1c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:51.548075 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-63296c33-e93b-4cfa-8174-f8a1ba817f1c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.186911 Oct 20 09:01:51.548743 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 104/189] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:51 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:52.081103 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b107fb5f-46e4-4619-b52d-4f014361324f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:52.082127 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b107fb5f-46e4-4619-b52d-4f014361324f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:52.088788 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b107fb5f-46e4-4619-b52d-4f014361324f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:52.089632 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b107fb5f-46e4-4619-b52d-4f014361324f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:52.090646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b107fb5f-46e4-4619-b52d-4f014361324f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:52.091606 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b107fb5f-46e4-4619-b52d-4f014361324f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:52.092276 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b107fb5f-46e4-4619-b52d-4f014361324f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:52.092945 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b107fb5f-46e4-4619-b52d-4f014361324f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:52.282342 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:52.283070 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:52.298679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:52.301079 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:52.301538 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:52.303290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:52.306406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:52.306406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:52.320623 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b107fb5f-46e4-4619-b52d-4f014361324f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:52.337401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] HTTP exception thrown: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Request method failure captured: Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb HTTP/1.1 Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ43Y8N0z31ZteV-DeaQvwlcZrLzMrdDddB29AIfHeNQv_Pc4-mn4P_uCxlT0wH7lMpGI-aeTtrIwSLpc0plDawMlJyffuY0pmMQ7B1Ur85GH54YqL0oJA2-ahXon_mGVDD30ODkTNkk0DzFSzA6e9J96U2rWcr7vLD2fp-RxSOT010 Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 4da29c9977ca44baa867b3c7fc86164b Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:01:52.342778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4da29c9977ca44baa867b3c7fc86164b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_4da29c9977ca44baa867b3c7fc86164b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/4da29c9977ca44baa867b3c7fc86164b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4da29c9977ca44baa867b3c7fc86164b"}]}] Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 4da29c9977ca44baa867b3c7fc86164b Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-DeleteServersTestJSON-1946604475-project Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: c7c40a4592874732871eaabd04d8662c Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-DeleteServersTestJSON-1946604475-project Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:52.344055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '91bccd9d-c1c7-477f-b8a5-914edd9fa2eb'} Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:01:52.344728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:52.346193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:52.347408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Returning 404 to user: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:01:52.347408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-da064889-5e30-40c4-a75b-b68753f13877 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 404 len: 111 microversion: 2.1 time: 0.066725 Oct 20 09:01:52.347408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 105/190] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:52 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 20 09:01:52.348368 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b107fb5f-46e4-4619-b52d-4f014361324f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.267751 Oct 20 09:01:52.359711 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 86/191] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:52 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:52.536486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:52.539440 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:52.552610 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:52.553507 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:52.554739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:52.557378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:52.558323 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:52.558935 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:52.568861 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=100 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:52.574838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0e8d3cfa-e9d1-4e2f-a4e3-ad57417bf673 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:52.575498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0e8d3cfa-e9d1-4e2f-a4e3-ad57417bf673 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:52.588194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0e8d3cfa-e9d1-4e2f-a4e3-ad57417bf673 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:52.588732 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0e8d3cfa-e9d1-4e2f-a4e3-ad57417bf673 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:52.589159 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0e8d3cfa-e9d1-4e2f-a4e3-ad57417bf673 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:52.589869 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0e8d3cfa-e9d1-4e2f-a4e3-ad57417bf673 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:52.590775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0e8d3cfa-e9d1-4e2f-a4e3-ad57417bf673 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:52.591405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0e8d3cfa-e9d1-4e2f-a4e3-ad57417bf673 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:52.598844 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] HTTP exception thrown: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Request method failure captured: Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb/os-volume_attachments HTTP/1.1 Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ43Y8N0z31ZteV-DeaQvwlcZrLzMrdDddB29AIfHeNQv_Pc4-mn4P_uCxlT0wH7lMpGI-aeTtrIwSLpc0plDawMlJyffuY0pmMQ7B1Ur85GH54YqL0oJA2-ahXon_mGVDD30ODkTNkk0DzFSzA6e9J96U2rWcr7vLD2fp-RxSOT010 Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 4da29c9977ca44baa867b3c7fc86164b Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:01:52.601011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4da29c9977ca44baa867b3c7fc86164b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_4da29c9977ca44baa867b3c7fc86164b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/4da29c9977ca44baa867b3c7fc86164b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4da29c9977ca44baa867b3c7fc86164b"}]}] Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 4da29c9977ca44baa867b3c7fc86164b Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-DeleteServersTestJSON-1946604475-project Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: c7c40a4592874732871eaabd04d8662c Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-DeleteServersTestJSON-1946604475-project Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:52.602234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'server_id': '91bccd9d-c1c7-477f-b8a5-914edd9fa2eb'} Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:01:52.602934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:52.604024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:52.606828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Returning 404 to user: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:01:52.608336 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9f582d0d-b8ab-48ed-ae2e-9ab72ce95998 tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.077725 Oct 20 09:01:52.609505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 106/192] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:01:52 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb/os-volume_attachments => generated 111 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:01:52.843995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0e8d3cfa-e9d1-4e2f-a4e3-ad57417bf673 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:52.857785 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0e8d3cfa-e9d1-4e2f-a4e3-ad57417bf673 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.287783 Oct 20 09:01:52.858386 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 87/193] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:52 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:53.375413 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-cc310154-1269-42bd-a76f-4bac76c4b754 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:53.376608 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-cc310154-1269-42bd-a76f-4bac76c4b754 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:53.383375 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cc310154-1269-42bd-a76f-4bac76c4b754 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:53.383855 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cc310154-1269-42bd-a76f-4bac76c4b754 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:53.384291 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cc310154-1269-42bd-a76f-4bac76c4b754 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:53.384987 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cc310154-1269-42bd-a76f-4bac76c4b754 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:53.385440 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cc310154-1269-42bd-a76f-4bac76c4b754 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:53.385861 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cc310154-1269-42bd-a76f-4bac76c4b754 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:53.542914 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-cc310154-1269-42bd-a76f-4bac76c4b754 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:53.552467 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-cc310154-1269-42bd-a76f-4bac76c4b754 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.181158 Oct 20 09:01:53.553025 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 107/194] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:53 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:53.884884 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ddeb23ce-3cba-47df-8761-81ab9e3b743e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:53.885511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ddeb23ce-3cba-47df-8761-81ab9e3b743e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:53.893627 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddeb23ce-3cba-47df-8761-81ab9e3b743e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:53.894560 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddeb23ce-3cba-47df-8761-81ab9e3b743e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:53.895496 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddeb23ce-3cba-47df-8761-81ab9e3b743e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:53.896132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddeb23ce-3cba-47df-8761-81ab9e3b743e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:53.896657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddeb23ce-3cba-47df-8761-81ab9e3b743e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:53.897109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddeb23ce-3cba-47df-8761-81ab9e3b743e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:53.954652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:53.955428 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:53.965883 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:53.966747 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:53.967379 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:53.968444 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:53.969082 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:53.969707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:53.988134 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] HTTP exception thrown: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Request method failure captured: Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: DELETE /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb HTTP/1.1 Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ43Y8N0z31ZteV-DeaQvwlcZrLzMrdDddB29AIfHeNQv_Pc4-mn4P_uCxlT0wH7lMpGI-aeTtrIwSLpc0plDawMlJyffuY0pmMQ7B1Ur85GH54YqL0oJA2-ahXon_mGVDD30ODkTNkk0DzFSzA6e9J96U2rWcr7vLD2fp-RxSOT010 Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 4da29c9977ca44baa867b3c7fc86164b Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:01:53.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4da29c9977ca44baa867b3c7fc86164b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_4da29c9977ca44baa867b3c7fc86164b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/4da29c9977ca44baa867b3c7fc86164b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4da29c9977ca44baa867b3c7fc86164b"}]}] Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 4da29c9977ca44baa867b3c7fc86164b Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-DeleteServersTestJSON-1946604475-project Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: c7c40a4592874732871eaabd04d8662c Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-DeleteServersTestJSON-1946604475-project Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:53.991824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '91bccd9d-c1c7-477f-b8a5-914edd9fa2eb'} Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:01:53.992465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:53.993499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:53.996172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Returning 404 to user: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:01:53.997079 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-22b1dbcf-3efd-4b6d-9a49-0a4a7fa36b9d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "DELETE /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 404 len: 111 microversion: 2.1 time: 0.047000 Oct 20 09:01:53.997714 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 108/195] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:01:53 2022] DELETE /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:01:54.007867 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:54.009724 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] [instance: 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:54.018335 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:54.019355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:54.020170 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:54.021152 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:54.022021 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:54.022832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:54.040538 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] HTTP exception thrown: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Request method failure captured: Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb HTTP/1.1 Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ43Y8N0z31ZteV-DeaQvwlcZrLzMrdDddB29AIfHeNQv_Pc4-mn4P_uCxlT0wH7lMpGI-aeTtrIwSLpc0plDawMlJyffuY0pmMQ7B1Ur85GH54YqL0oJA2-ahXon_mGVDD30ODkTNkk0DzFSzA6e9J96U2rWcr7vLD2fp-RxSOT010 Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 4da29c9977ca44baa867b3c7fc86164b Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:01:54.042415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4da29c9977ca44baa867b3c7fc86164b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_4da29c9977ca44baa867b3c7fc86164b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/4da29c9977ca44baa867b3c7fc86164b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4da29c9977ca44baa867b3c7fc86164b"}]}] Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 4da29c9977ca44baa867b3c7fc86164b Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-DeleteServersTestJSON-1946604475 Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-DeleteServersTestJSON-1946604475-project Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: c7c40a4592874732871eaabd04d8662c Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-DeleteServersTestJSON-1946604475-project Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:54.043574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '91bccd9d-c1c7-477f-b8a5-914edd9fa2eb'} Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:01:54.044285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. Oct 20 09:01:54.045350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:01:54.048223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] Returning 404 to user: Instance 91bccd9d-c1c7-477f-b8a5-914edd9fa2eb could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:01:54.049389 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f86f01d9-65e6-42a7-8d15-b542a195182d tempest-DeleteServersTestJSON-1946604475 tempest-DeleteServersTestJSON-1946604475-project] 10.222.0.36 "GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb" status: 404 len: 111 microversion: 2.1 time: 0.047413 Oct 20 09:01:54.050511 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 109/196] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:54 2022] GET /compute/v2.1/servers/91bccd9d-c1c7-477f-b8a5-914edd9fa2eb => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:01:54.165944 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-86dbf8e4-150d-41ca-ac16-c7624c07cd47 req-66597a52-c411-4045-9d11-23b7154eb00d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e67be6e-328c-4a4f-8a2b-083b26c3d62a", "name": "network-vif-plugged", "status": "completed", "tag": "a052a7b6-dd87-4142-8f12-b3b7205a2cef"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:54.178182 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-86dbf8e4-150d-41ca-ac16-c7624c07cd47 req-66597a52-c411-4045-9d11-23b7154eb00d service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:54.179898 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-86dbf8e4-150d-41ca-ac16-c7624c07cd47 req-66597a52-c411-4045-9d11-23b7154eb00d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:54.180497 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-86dbf8e4-150d-41ca-ac16-c7624c07cd47 req-66597a52-c411-4045-9d11-23b7154eb00d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:54.196829 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-86dbf8e4-150d-41ca-ac16-c7624c07cd47 req-66597a52-c411-4045-9d11-23b7154eb00d service nova] Creating event network-vif-plugged:a052a7b6-dd87-4142-8f12-b3b7205a2cef for instance 8e67be6e-328c-4a4f-8a2b-083b26c3d62a on hv-cinder-maste Oct 20 09:01:54.201809 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ddeb23ce-3cba-47df-8761-81ab9e3b743e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:54.203448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-86dbf8e4-150d-41ca-ac16-c7624c07cd47 req-66597a52-c411-4045-9d11-23b7154eb00d service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043166 Oct 20 09:01:54.204777 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 110/197] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:01:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:01:54.214282 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ddeb23ce-3cba-47df-8761-81ab9e3b743e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.334654 Oct 20 09:01:54.215601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 88/198] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:53 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:54.504606 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1810431159", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "networks": [{"uuid": "7fdc4269-1e8c-4b8f-884b-33c74a51bd63"}]}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:01:54.567621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] validate_networks() for [('7fdc4269-1e8c-4b8f-884b-33c74a51bd63', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:01:54.576171 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bf1882f5-ea1b-443c-bce4-87d843fedccf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:54.577132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bf1882f5-ea1b-443c-bce4-87d843fedccf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:54.584111 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf1882f5-ea1b-443c-bce4-87d843fedccf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:54.584691 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf1882f5-ea1b-443c-bce4-87d843fedccf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:54.585398 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf1882f5-ea1b-443c-bce4-87d843fedccf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:54.586174 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf1882f5-ea1b-443c-bce4-87d843fedccf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:54.587015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf1882f5-ea1b-443c-bce4-87d843fedccf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:54.587824 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf1882f5-ea1b-443c-bce4-87d843fedccf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:54.958096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bf1882f5-ea1b-443c-bce4-87d843fedccf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:54.969782 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bf1882f5-ea1b-443c-bce4-87d843fedccf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.397323 Oct 20 09:01:54.970837 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 89/199] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:54 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:55.072658 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:01:55.073059 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:01:55.073619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:01:55.075638 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:01:55.075638 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:01:55.084470 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:55.085150 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:55.086147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:55.086982 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:55.087513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:55.088117 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:55.113366 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Getting quotas for project 9dc1a3b80bd34c8e8f691a9b47457f35. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:01:55.119378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Getting quotas for user 7e0951446ad449d6b725fe6cc14df1f2 and project 9dc1a3b80bd34c8e8f691a9b47457f35. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:01:55.180157 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:01:55.183544 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:01:55.240975 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f3570246-ee04-4950-a87e-21615b1010a7 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:55.242126 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f3570246-ee04-4950-a87e-21615b1010a7 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:55.255390 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f3570246-ee04-4950-a87e-21615b1010a7 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:55.262177 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f3570246-ee04-4950-a87e-21615b1010a7 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:55.262669 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f3570246-ee04-4950-a87e-21615b1010a7 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:55.263508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f3570246-ee04-4950-a87e-21615b1010a7 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:55.264197 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f3570246-ee04-4950-a87e-21615b1010a7 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:55.264806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f3570246-ee04-4950-a87e-21615b1010a7 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:55.275531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-63c19ad9-a2d7-4749-9176-d81e210b9620 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.775638 Oct 20 09:01:55.277451 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 111/200] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:01:54 2022] POST /compute/v2.1/servers => generated 380 bytes in 778 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:01:55.290213 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1a2ed209-4b14-4670-ae3a-2d61ed655d8b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:55.291351 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1a2ed209-4b14-4670-ae3a-2d61ed655d8b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:55.411633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1a2ed209-4b14-4670-ae3a-2d61ed655d8b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:55.419312 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1a2ed209-4b14-4670-ae3a-2d61ed655d8b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1219 microversion: 2.1 time: 0.134075 Oct 20 09:01:55.423943 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 112/201] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:55 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1219 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:55.558396 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f3570246-ee04-4950-a87e-21615b1010a7 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:55.572482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f3570246-ee04-4950-a87e-21615b1010a7 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.334382 Oct 20 09:01:55.573412 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 90/202] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:55 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:55.993510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fb6a5464-cba8-41b4-841e-8ed958f734e3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:55.994868 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fb6a5464-cba8-41b4-841e-8ed958f734e3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:56.027227 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fb6a5464-cba8-41b4-841e-8ed958f734e3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:56.037154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fb6a5464-cba8-41b4-841e-8ed958f734e3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:56.039135 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fb6a5464-cba8-41b4-841e-8ed958f734e3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:56.044870 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fb6a5464-cba8-41b4-841e-8ed958f734e3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:56.045560 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fb6a5464-cba8-41b4-841e-8ed958f734e3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:56.047093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fb6a5464-cba8-41b4-841e-8ed958f734e3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:56.239064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-fb6a5464-cba8-41b4-841e-8ed958f734e3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:56.253804 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-fb6a5464-cba8-41b4-841e-8ed958f734e3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.262257 Oct 20 09:01:56.254322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 113/203] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:55 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:56.448185 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-85bfd4b9-ab35-45b2-861a-dce30b7b666a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:56.449999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-85bfd4b9-ab35-45b2-861a-dce30b7b666a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:56.457765 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85bfd4b9-ab35-45b2-861a-dce30b7b666a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:56.458803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85bfd4b9-ab35-45b2-861a-dce30b7b666a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:56.459685 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85bfd4b9-ab35-45b2-861a-dce30b7b666a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:56.461181 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85bfd4b9-ab35-45b2-861a-dce30b7b666a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:56.461799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85bfd4b9-ab35-45b2-861a-dce30b7b666a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:56.462380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85bfd4b9-ab35-45b2-861a-dce30b7b666a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:56.571441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-85bfd4b9-ab35-45b2-861a-dce30b7b666a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:56.580001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-85bfd4b9-ab35-45b2-861a-dce30b7b666a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1223 microversion: 2.1 time: 0.138479 Oct 20 09:01:56.580961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 91/204] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:56 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1223 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:56.600810 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0d1e6748-ef97-4f7a-941d-8ddc90f77135 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:56.601672 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0d1e6748-ef97-4f7a-941d-8ddc90f77135 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:56.609848 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d1e6748-ef97-4f7a-941d-8ddc90f77135 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:56.610589 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d1e6748-ef97-4f7a-941d-8ddc90f77135 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:56.611640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d1e6748-ef97-4f7a-941d-8ddc90f77135 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:56.612349 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d1e6748-ef97-4f7a-941d-8ddc90f77135 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:56.612806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d1e6748-ef97-4f7a-941d-8ddc90f77135 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:56.613228 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d1e6748-ef97-4f7a-941d-8ddc90f77135 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:56.978677 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0d1e6748-ef97-4f7a-941d-8ddc90f77135 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:56.986272 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0d1e6748-ef97-4f7a-941d-8ddc90f77135 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1314 microversion: 2.1 time: 0.389652 Oct 20 09:01:57.004548 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 114/205] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:56 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1314 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:57.278578 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a815c7fd-2f50-4550-bfeb-e98fae3d3b70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:57.279524 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a815c7fd-2f50-4550-bfeb-e98fae3d3b70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:57.290071 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a815c7fd-2f50-4550-bfeb-e98fae3d3b70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:57.290591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a815c7fd-2f50-4550-bfeb-e98fae3d3b70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:57.292490 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a815c7fd-2f50-4550-bfeb-e98fae3d3b70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:57.293218 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a815c7fd-2f50-4550-bfeb-e98fae3d3b70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:57.293680 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a815c7fd-2f50-4550-bfeb-e98fae3d3b70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:57.294117 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a815c7fd-2f50-4550-bfeb-e98fae3d3b70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:57.606525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9d0bf8e4-da7e-463c-ba37-4432f25f65d1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:57.607757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9d0bf8e4-da7e-463c-ba37-4432f25f65d1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:57.616729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d0bf8e4-da7e-463c-ba37-4432f25f65d1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:57.617112 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d0bf8e4-da7e-463c-ba37-4432f25f65d1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:57.617707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d0bf8e4-da7e-463c-ba37-4432f25f65d1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:57.618767 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d0bf8e4-da7e-463c-ba37-4432f25f65d1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:57.619512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d0bf8e4-da7e-463c-ba37-4432f25f65d1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:57.620280 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d0bf8e4-da7e-463c-ba37-4432f25f65d1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:57.648626 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a815c7fd-2f50-4550-bfeb-e98fae3d3b70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:57.658092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a815c7fd-2f50-4550-bfeb-e98fae3d3b70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.383920 Oct 20 09:01:57.659990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 92/206] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:57 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:57.720802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9d0bf8e4-da7e-463c-ba37-4432f25f65d1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:57.730728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9d0bf8e4-da7e-463c-ba37-4432f25f65d1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1271 microversion: 2.1 time: 0.131080 Oct 20 09:01:57.731671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 115/207] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:57 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1271 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:01:58.011432 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-78ac6e0c-e367-4050-8ff9-589258dbe7d2 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:58.012529 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-78ac6e0c-e367-4050-8ff9-589258dbe7d2 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:58.027787 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-78ac6e0c-e367-4050-8ff9-589258dbe7d2 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:58.028447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-78ac6e0c-e367-4050-8ff9-589258dbe7d2 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:58.029036 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-78ac6e0c-e367-4050-8ff9-589258dbe7d2 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:58.030285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-78ac6e0c-e367-4050-8ff9-589258dbe7d2 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:58.030927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-78ac6e0c-e367-4050-8ff9-589258dbe7d2 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:58.031499 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-78ac6e0c-e367-4050-8ff9-589258dbe7d2 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:58.303707 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-78ac6e0c-e367-4050-8ff9-589258dbe7d2 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:58.322189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-78ac6e0c-e367-4050-8ff9-589258dbe7d2 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1499 microversion: 2.1 time: 0.316567 Oct 20 09:01:58.323609 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 93/208] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:58 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1499 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:58.681879 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-33cea9f9-5e00-4968-812b-8e59d48177f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:58.682510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-33cea9f9-5e00-4968-812b-8e59d48177f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:58.689231 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33cea9f9-5e00-4968-812b-8e59d48177f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:58.689714 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33cea9f9-5e00-4968-812b-8e59d48177f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:58.691707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33cea9f9-5e00-4968-812b-8e59d48177f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:58.692623 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33cea9f9-5e00-4968-812b-8e59d48177f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:58.693333 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33cea9f9-5e00-4968-812b-8e59d48177f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:58.694027 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33cea9f9-5e00-4968-812b-8e59d48177f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:58.755332 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-886bc141-2528-4b97-bce6-c4c555156a85 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:58.757173 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-886bc141-2528-4b97-bce6-c4c555156a85 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:58.783325 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-886bc141-2528-4b97-bce6-c4c555156a85 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:58.783845 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-886bc141-2528-4b97-bce6-c4c555156a85 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:58.784281 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-886bc141-2528-4b97-bce6-c4c555156a85 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:58.799492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-886bc141-2528-4b97-bce6-c4c555156a85 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:58.800237 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-886bc141-2528-4b97-bce6-c4c555156a85 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:58.800858 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-886bc141-2528-4b97-bce6-c4c555156a85 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:58.922029 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-33cea9f9-5e00-4968-812b-8e59d48177f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:58.934867 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-33cea9f9-5e00-4968-812b-8e59d48177f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.257194 Oct 20 09:01:58.936072 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 116/209] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:58 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:58.987851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-886bc141-2528-4b97-bce6-c4c555156a85 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:58.997939 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-886bc141-2528-4b97-bce6-c4c555156a85 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1277 microversion: 2.1 time: 0.247737 Oct 20 09:01:58.999066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 94/210] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:58 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1277 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:59.250827 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1d564238-a978-4bda-80e3-6c560a223949 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:59.251315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1d564238-a978-4bda-80e3-6c560a223949 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:59.257530 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d564238-a978-4bda-80e3-6c560a223949 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:59.258247 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d564238-a978-4bda-80e3-6c560a223949 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:59.258915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d564238-a978-4bda-80e3-6c560a223949 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:59.259795 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d564238-a978-4bda-80e3-6c560a223949 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:59.260429 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d564238-a978-4bda-80e3-6c560a223949 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:59.261031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d564238-a978-4bda-80e3-6c560a223949 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:59.290166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1d564238-a978-4bda-80e3-6c560a223949 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:01:59.386700 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:01:59.412837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1d564238-a978-4bda-80e3-6c560a223949 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "DELETE /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 204 len: 0 microversion: 2.1 time: 0.165720 Oct 20 09:01:59.414464 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 117/211] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:01:59 2022] DELETE /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:01:59.423257 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cc97d18f-bf71-4bef-b44e-f31daa1d18e1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:59.424446 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-cc97d18f-bf71-4bef-b44e-f31daa1d18e1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:59.434506 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc97d18f-bf71-4bef-b44e-f31daa1d18e1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:59.434997 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc97d18f-bf71-4bef-b44e-f31daa1d18e1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:59.435431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc97d18f-bf71-4bef-b44e-f31daa1d18e1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:59.436120 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc97d18f-bf71-4bef-b44e-f31daa1d18e1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:59.436565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc97d18f-bf71-4bef-b44e-f31daa1d18e1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:59.436985 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc97d18f-bf71-4bef-b44e-f31daa1d18e1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:59.819934 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-cc97d18f-bf71-4bef-b44e-f31daa1d18e1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:01:59.830605 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-cc97d18f-bf71-4bef-b44e-f31daa1d18e1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1505 microversion: 2.1 time: 0.411665 Oct 20 09:01:59.831876 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 95/212] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:59 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1505 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:01:59.965257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8a532a37-7861-48b9-a001-fb5dd234c5ad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:01:59.965994 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8a532a37-7861-48b9-a001-fb5dd234c5ad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:01:59.974072 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8a532a37-7861-48b9-a001-fb5dd234c5ad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:59.974666 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8a532a37-7861-48b9-a001-fb5dd234c5ad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:59.975645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8a532a37-7861-48b9-a001-fb5dd234c5ad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:01:59.976553 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8a532a37-7861-48b9-a001-fb5dd234c5ad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:01:59.977023 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8a532a37-7861-48b9-a001-fb5dd234c5ad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:01:59.977626 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8a532a37-7861-48b9-a001-fb5dd234c5ad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:00.024669 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-330dd11a-5a6e-41e7-bd8a-e58291e1c6dd tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:00.025529 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-330dd11a-5a6e-41e7-bd8a-e58291e1c6dd tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:00.034021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-330dd11a-5a6e-41e7-bd8a-e58291e1c6dd tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:00.034974 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-330dd11a-5a6e-41e7-bd8a-e58291e1c6dd tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:00.035743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-330dd11a-5a6e-41e7-bd8a-e58291e1c6dd tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:00.036850 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-330dd11a-5a6e-41e7-bd8a-e58291e1c6dd tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:00.037626 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-330dd11a-5a6e-41e7-bd8a-e58291e1c6dd tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:00.052531 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-330dd11a-5a6e-41e7-bd8a-e58291e1c6dd tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:00.149520 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8a532a37-7861-48b9-a001-fb5dd234c5ad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:00.158409 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8a532a37-7861-48b9-a001-fb5dd234c5ad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.198115 Oct 20 09:02:00.159035 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 118/213] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:01:59 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:00.454226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-330dd11a-5a6e-41e7-bd8a-e58291e1c6dd tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:00.462834 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-330dd11a-5a6e-41e7-bd8a-e58291e1c6dd tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.443499 Oct 20 09:02:00.463398 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 96/214] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:00 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:00.859685 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-53732366-5478-4db6-8b50-6ccdd338b098 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:00.860285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-53732366-5478-4db6-8b50-6ccdd338b098 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:00.865988 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53732366-5478-4db6-8b50-6ccdd338b098 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:00.867096 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53732366-5478-4db6-8b50-6ccdd338b098 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:00.867519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53732366-5478-4db6-8b50-6ccdd338b098 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:00.868194 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53732366-5478-4db6-8b50-6ccdd338b098 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:00.868635 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53732366-5478-4db6-8b50-6ccdd338b098 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:00.869074 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-53732366-5478-4db6-8b50-6ccdd338b098 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:01.010852 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-53732366-5478-4db6-8b50-6ccdd338b098 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:01.018817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-53732366-5478-4db6-8b50-6ccdd338b098 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1505 microversion: 2.1 time: 0.162869 Oct 20 09:02:01.020552 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 119/215] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:00 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1505 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:01.182950 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-455de928-1e8e-4a0a-9fe8-c479636dda18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:01.184146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-455de928-1e8e-4a0a-9fe8-c479636dda18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:01.189952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-455de928-1e8e-4a0a-9fe8-c479636dda18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:01.190435 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-455de928-1e8e-4a0a-9fe8-c479636dda18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:01.191421 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-455de928-1e8e-4a0a-9fe8-c479636dda18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:01.192117 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-455de928-1e8e-4a0a-9fe8-c479636dda18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:01.192571 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-455de928-1e8e-4a0a-9fe8-c479636dda18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:01.192999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-455de928-1e8e-4a0a-9fe8-c479636dda18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:01.333676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-455de928-1e8e-4a0a-9fe8-c479636dda18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:01.343449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-455de928-1e8e-4a0a-9fe8-c479636dda18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.165050 Oct 20 09:02:01.344160 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 97/216] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:01 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:01.486605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0bf350d8-1712-47bc-a04f-bccbda8ad547 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:01.487176 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0bf350d8-1712-47bc-a04f-bccbda8ad547 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:01.493815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf350d8-1712-47bc-a04f-bccbda8ad547 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:01.494294 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf350d8-1712-47bc-a04f-bccbda8ad547 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:01.494758 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf350d8-1712-47bc-a04f-bccbda8ad547 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:01.495446 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf350d8-1712-47bc-a04f-bccbda8ad547 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:01.495892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf350d8-1712-47bc-a04f-bccbda8ad547 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:01.496309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bf350d8-1712-47bc-a04f-bccbda8ad547 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:01.673055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0bf350d8-1712-47bc-a04f-bccbda8ad547 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:01.682957 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0bf350d8-1712-47bc-a04f-bccbda8ad547 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.200254 Oct 20 09:02:01.684064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 120/217] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:01 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:02.053336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3e142549-a4a0-45b7-b2c9-b0ecf0fbe66b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:02.054343 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3e142549-a4a0-45b7-b2c9-b0ecf0fbe66b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:02.063162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e142549-a4a0-45b7-b2c9-b0ecf0fbe66b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:02.063814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e142549-a4a0-45b7-b2c9-b0ecf0fbe66b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:02.064366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e142549-a4a0-45b7-b2c9-b0ecf0fbe66b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:02.065330 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e142549-a4a0-45b7-b2c9-b0ecf0fbe66b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:02.065883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e142549-a4a0-45b7-b2c9-b0ecf0fbe66b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:02.066557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e142549-a4a0-45b7-b2c9-b0ecf0fbe66b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:02.343001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3e142549-a4a0-45b7-b2c9-b0ecf0fbe66b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:02.345603 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:02.353840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3e142549-a4a0-45b7-b2c9-b0ecf0fbe66b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1505 microversion: 2.1 time: 0.304999 Oct 20 09:02:02.354427 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 98/218] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:02 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1505 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:02.369267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ce01b502-42e5-4ea9-80f1-09c13704bc84 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:02.370602 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ce01b502-42e5-4ea9-80f1-09c13704bc84 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:02.379257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce01b502-42e5-4ea9-80f1-09c13704bc84 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:02.380065 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce01b502-42e5-4ea9-80f1-09c13704bc84 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:02.380678 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce01b502-42e5-4ea9-80f1-09c13704bc84 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:02.382442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce01b502-42e5-4ea9-80f1-09c13704bc84 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:02.383405 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce01b502-42e5-4ea9-80f1-09c13704bc84 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:02.384694 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ce01b502-42e5-4ea9-80f1-09c13704bc84 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:02.540454 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ce01b502-42e5-4ea9-80f1-09c13704bc84 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:02.549270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ce01b502-42e5-4ea9-80f1-09c13704bc84 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.184708 Oct 20 09:02:02.550837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 121/219] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:02 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:02.712840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c67dc836-0e16-4dd4-b977-025c725df39f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:02.713474 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c67dc836-0e16-4dd4-b977-025c725df39f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:02.738260 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c67dc836-0e16-4dd4-b977-025c725df39f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:02.739130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c67dc836-0e16-4dd4-b977-025c725df39f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:02.739728 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c67dc836-0e16-4dd4-b977-025c725df39f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:02.740588 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c67dc836-0e16-4dd4-b977-025c725df39f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:02.741796 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c67dc836-0e16-4dd4-b977-025c725df39f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:02.742380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c67dc836-0e16-4dd4-b977-025c725df39f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:03.110932 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c67dc836-0e16-4dd4-b977-025c725df39f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:03.119196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c67dc836-0e16-4dd4-b977-025c725df39f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.417438 Oct 20 09:02:03.119698 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 99/220] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:02 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:03.378018 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7cb449e2-f09b-475b-93c5-b56751e2d950 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:03.379338 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7cb449e2-f09b-475b-93c5-b56751e2d950 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:03.388111 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7cb449e2-f09b-475b-93c5-b56751e2d950 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:03.388700 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7cb449e2-f09b-475b-93c5-b56751e2d950 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:03.389198 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7cb449e2-f09b-475b-93c5-b56751e2d950 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:03.389968 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7cb449e2-f09b-475b-93c5-b56751e2d950 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:03.390531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7cb449e2-f09b-475b-93c5-b56751e2d950 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:03.391022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7cb449e2-f09b-475b-93c5-b56751e2d950 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:03.545766 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7cb449e2-f09b-475b-93c5-b56751e2d950 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:03.558983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7cb449e2-f09b-475b-93c5-b56751e2d950 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1505 microversion: 2.1 time: 0.185347 Oct 20 09:02:03.562026 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 122/221] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:03 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1505 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:03.579625 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-65c5217c-8bab-497e-8a2d-5f8938ca8c4a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:03.580242 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-65c5217c-8bab-497e-8a2d-5f8938ca8c4a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:03.587327 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65c5217c-8bab-497e-8a2d-5f8938ca8c4a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:03.587828 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65c5217c-8bab-497e-8a2d-5f8938ca8c4a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:03.588257 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65c5217c-8bab-497e-8a2d-5f8938ca8c4a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:03.589189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65c5217c-8bab-497e-8a2d-5f8938ca8c4a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:03.589383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65c5217c-8bab-497e-8a2d-5f8938ca8c4a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:03.589803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65c5217c-8bab-497e-8a2d-5f8938ca8c4a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:04.145451 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-16e7660f-7adc-44c4-90bd-830d37c55817 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:04.146295 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-16e7660f-7adc-44c4-90bd-830d37c55817 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:04.152841 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e7660f-7adc-44c4-90bd-830d37c55817 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:04.153291 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e7660f-7adc-44c4-90bd-830d37c55817 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:04.153726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e7660f-7adc-44c4-90bd-830d37c55817 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:04.154526 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e7660f-7adc-44c4-90bd-830d37c55817 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:04.155014 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e7660f-7adc-44c4-90bd-830d37c55817 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:04.155396 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e7660f-7adc-44c4-90bd-830d37c55817 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:04.227898 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65c5217c-8bab-497e-8a2d-5f8938ca8c4a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:04.236507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-65c5217c-8bab-497e-8a2d-5f8938ca8c4a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.667053 Oct 20 09:02:04.241917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 100/222] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:03 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:04.295877 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-16e7660f-7adc-44c4-90bd-830d37c55817 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:04.303789 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-16e7660f-7adc-44c4-90bd-830d37c55817 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.162946 Oct 20 09:02:04.304457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 123/223] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:04 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:04.586192 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ac2dafe8-0ce4-4542-aa9c-da0d91e1975b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:04.587040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ac2dafe8-0ce4-4542-aa9c-da0d91e1975b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:04.593830 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ac2dafe8-0ce4-4542-aa9c-da0d91e1975b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:04.594317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ac2dafe8-0ce4-4542-aa9c-da0d91e1975b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:04.594807 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ac2dafe8-0ce4-4542-aa9c-da0d91e1975b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:04.595500 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ac2dafe8-0ce4-4542-aa9c-da0d91e1975b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:04.595966 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ac2dafe8-0ce4-4542-aa9c-da0d91e1975b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:04.596371 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ac2dafe8-0ce4-4542-aa9c-da0d91e1975b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:04.723628 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ac2dafe8-0ce4-4542-aa9c-da0d91e1975b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:04.731234 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ac2dafe8-0ce4-4542-aa9c-da0d91e1975b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1463 microversion: 2.1 time: 0.148917 Oct 20 09:02:04.731798 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 101/224] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:04 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1463 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:04.781606 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-b95e6a37-6d40-4069-8910-bd07f7a6de76 req-6eedc006-6cdf-44c8-9bbb-27ee907e00f6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e67be6e-328c-4a4f-8a2b-083b26c3d62a", "name": "network-vif-deleted", "tag": "a052a7b6-dd87-4142-8f12-b3b7205a2cef"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:04.792235 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-b95e6a37-6d40-4069-8910-bd07f7a6de76 req-6eedc006-6cdf-44c8-9bbb-27ee907e00f6 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:04.795175 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-b95e6a37-6d40-4069-8910-bd07f7a6de76 req-6eedc006-6cdf-44c8-9bbb-27ee907e00f6 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:04.795777 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-b95e6a37-6d40-4069-8910-bd07f7a6de76 req-6eedc006-6cdf-44c8-9bbb-27ee907e00f6 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:04.809581 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-b95e6a37-6d40-4069-8910-bd07f7a6de76 req-6eedc006-6cdf-44c8-9bbb-27ee907e00f6 service nova] Creating event network-vif-deleted:a052a7b6-dd87-4142-8f12-b3b7205a2cef for instance 8e67be6e-328c-4a4f-8a2b-083b26c3d62a on hv-cinder-maste Oct 20 09:02:04.814741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-b95e6a37-6d40-4069-8910-bd07f7a6de76 req-6eedc006-6cdf-44c8-9bbb-27ee907e00f6 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038238 Oct 20 09:02:04.815447 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 124/225] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:02:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:02:05.260121 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-38b247cd-46de-452e-bf00-8a87ed8ad6ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:05.260890 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-38b247cd-46de-452e-bf00-8a87ed8ad6ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:05.267383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38b247cd-46de-452e-bf00-8a87ed8ad6ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:05.267952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38b247cd-46de-452e-bf00-8a87ed8ad6ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:05.268458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38b247cd-46de-452e-bf00-8a87ed8ad6ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:05.269225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38b247cd-46de-452e-bf00-8a87ed8ad6ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:05.270294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38b247cd-46de-452e-bf00-8a87ed8ad6ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:05.270835 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38b247cd-46de-452e-bf00-8a87ed8ad6ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:05.324555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:05.328888 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-21f9b4bd-9993-4273-a99f-6c450deec723 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:05.329658 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-21f9b4bd-9993-4273-a99f-6c450deec723 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:05.336654 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21f9b4bd-9993-4273-a99f-6c450deec723 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:05.337210 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21f9b4bd-9993-4273-a99f-6c450deec723 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:05.337786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21f9b4bd-9993-4273-a99f-6c450deec723 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:05.338573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21f9b4bd-9993-4273-a99f-6c450deec723 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:05.339241 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21f9b4bd-9993-4273-a99f-6c450deec723 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:05.339920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21f9b4bd-9993-4273-a99f-6c450deec723 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:05.419831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=8 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:05.473308 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-38b247cd-46de-452e-bf00-8a87ed8ad6ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:05.488980 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-38b247cd-46de-452e-bf00-8a87ed8ad6ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.233458 Oct 20 09:02:05.490013 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 102/226] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:05 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:05.495042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-21f9b4bd-9993-4273-a99f-6c450deec723 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:05.506026 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-21f9b4bd-9993-4273-a99f-6c450deec723 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.182264 Oct 20 09:02:05.507599 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 125/227] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:05 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:05.755593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e4b7429a-8582-4119-bc89-31290e391113 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:05.756344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e4b7429a-8582-4119-bc89-31290e391113 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:05.763433 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e4b7429a-8582-4119-bc89-31290e391113 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:05.764147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e4b7429a-8582-4119-bc89-31290e391113 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:05.765313 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e4b7429a-8582-4119-bc89-31290e391113 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:05.766241 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e4b7429a-8582-4119-bc89-31290e391113 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:05.766922 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e4b7429a-8582-4119-bc89-31290e391113 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:05.767547 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e4b7429a-8582-4119-bc89-31290e391113 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:05.962103 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e4b7429a-8582-4119-bc89-31290e391113 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:05.970778 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e4b7429a-8582-4119-bc89-31290e391113 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1463 microversion: 2.1 time: 0.219097 Oct 20 09:02:05.986767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 103/228] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:05 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1463 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:06.518593 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-cf74d091-0c1e-47bd-9721-42dfea699a6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:06.519210 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-cf74d091-0c1e-47bd-9721-42dfea699a6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:06.532920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cf74d091-0c1e-47bd-9721-42dfea699a6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:06.533513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6328c961-0b7d-483d-8d36-a73520c64514 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:06.533785 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cf74d091-0c1e-47bd-9721-42dfea699a6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:06.534126 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6328c961-0b7d-483d-8d36-a73520c64514 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:06.534571 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cf74d091-0c1e-47bd-9721-42dfea699a6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:06.535439 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cf74d091-0c1e-47bd-9721-42dfea699a6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:06.536033 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cf74d091-0c1e-47bd-9721-42dfea699a6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:06.541913 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cf74d091-0c1e-47bd-9721-42dfea699a6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:06.543654 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6328c961-0b7d-483d-8d36-a73520c64514 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:06.545915 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6328c961-0b7d-483d-8d36-a73520c64514 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:06.546650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6328c961-0b7d-483d-8d36-a73520c64514 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:06.547532 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6328c961-0b7d-483d-8d36-a73520c64514 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:06.548146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6328c961-0b7d-483d-8d36-a73520c64514 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:06.548866 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6328c961-0b7d-483d-8d36-a73520c64514 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:06.555642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:06.710925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-cf74d091-0c1e-47bd-9721-42dfea699a6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:06.723236 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-cf74d091-0c1e-47bd-9721-42dfea699a6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.213201 Oct 20 09:02:06.724179 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 126/229] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:06 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:06.773713 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6328c961-0b7d-483d-8d36-a73520c64514 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:06.783050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6328c961-0b7d-483d-8d36-a73520c64514 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.253157 Oct 20 09:02:06.783636 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 104/230] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:06 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:06.852771 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-ae863db4-69f9-4842-81c0-aa8f54fe2b95 req-93ee479d-aa23-4ecd-af76-0ba42e38c089 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fe511cdb-e20c-4ec9-a37b-1eb239d3ed67", "tag": "6bd83ee3-a5b5-413b-9b18-8034518260a1"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:06.875533 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-ae863db4-69f9-4842-81c0-aa8f54fe2b95 req-93ee479d-aa23-4ecd-af76-0ba42e38c089 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:06.876241 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-ae863db4-69f9-4842-81c0-aa8f54fe2b95 req-93ee479d-aa23-4ecd-af76-0ba42e38c089 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:06.876445 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-ae863db4-69f9-4842-81c0-aa8f54fe2b95 req-93ee479d-aa23-4ecd-af76-0ba42e38c089 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:06.888531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-ae863db4-69f9-4842-81c0-aa8f54fe2b95 req-93ee479d-aa23-4ecd-af76-0ba42e38c089 service nova] Creating event network-changed:6bd83ee3-a5b5-413b-9b18-8034518260a1 for instance fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 on hv-cinder-maste Oct 20 09:02:06.893188 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-ae863db4-69f9-4842-81c0-aa8f54fe2b95 req-93ee479d-aa23-4ecd-af76-0ba42e38c089 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046082 Oct 20 09:02:06.894315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 127/231] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:02:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:02:06.994018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:06.994664 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:07.003685 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:07.004449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:07.005047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:07.005902 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:07.006527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:07.007135 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:07.069537 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:07.070116 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:07.070655 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:07.071315 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lazy-loading 'fault' on Instance uuid 8e67be6e-328c-4a4f-8a2b-083b26c3d62a {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:02:07.164759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:07.183075 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0267bf33-df73-402e-8f72-4e37b2898eaa tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 200 len: 1304 microversion: 2.1 time: 0.193238 Oct 20 09:02:07.183932 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 105/232] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:06 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 1304 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:07.315992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ef24e99b-aff4-4041-b4ad-606da299c7b3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'create', calling method: .version_select at 0x7f5afbae0f70>, body: {"keypair": {"name": "tempest-keypair-944100248", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC+jPRU/sLU9DOKozoeBDvDrv0gdf12DAs3e8zNMfDwqVjZgw5G3chx5brzHCwVQV5UGcuAbBkwyy1CscKJXR9/p5vVSxwvdpcnXmEysCKOjd2guJV+Thnah4UM1talaBA=="}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:07.335172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-ef24e99b-aff4-4041-b4ad-606da299c7b3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Getting quotas for project f1061324903c42ed9f0306471a113b82. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:02:07.342422 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-ef24e99b-aff4-4041-b4ad-606da299c7b3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Getting quotas for user be1f82d7a0a84fbebdd0dffef9f098ea and project f1061324903c42ed9f0306471a113b82. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:02:07.365627 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ef24e99b-aff4-4041-b4ad-606da299c7b3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.185510 Oct 20 09:02:07.367049 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 128/233] 10.222.0.36 () {60 vars in 1159 bytes} [Thu Oct 20 09:02:07 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:02:07.749652 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7b09aa4c-8058-4ad3-8ca3-f6cf1bf71470 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:07.750746 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7b09aa4c-8058-4ad3-8ca3-f6cf1bf71470 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:07.759289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b09aa4c-8058-4ad3-8ca3-f6cf1bf71470 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:07.760158 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b09aa4c-8058-4ad3-8ca3-f6cf1bf71470 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:07.760845 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b09aa4c-8058-4ad3-8ca3-f6cf1bf71470 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:07.762760 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b09aa4c-8058-4ad3-8ca3-f6cf1bf71470 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:07.763976 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b09aa4c-8058-4ad3-8ca3-f6cf1bf71470 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:07.764616 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7b09aa4c-8058-4ad3-8ca3-f6cf1bf71470 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:07.806843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-164c05b7-91f3-4f86-aa19-dae3c524ce0f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:07.818424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-164c05b7-91f3-4f86-aa19-dae3c524ce0f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:07.830745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-164c05b7-91f3-4f86-aa19-dae3c524ce0f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:07.831703 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-164c05b7-91f3-4f86-aa19-dae3c524ce0f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:07.832509 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-164c05b7-91f3-4f86-aa19-dae3c524ce0f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:07.833384 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-164c05b7-91f3-4f86-aa19-dae3c524ce0f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:07.833837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-164c05b7-91f3-4f86-aa19-dae3c524ce0f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:07.834268 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-164c05b7-91f3-4f86-aa19-dae3c524ce0f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:07.957143 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7b09aa4c-8058-4ad3-8ca3-f6cf1bf71470 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:07.979949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7b09aa4c-8058-4ad3-8ca3-f6cf1bf71470 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.237172 Oct 20 09:02:07.983582 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 106/234] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:07 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:08.075515 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-164c05b7-91f3-4f86-aa19-dae3c524ce0f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:08.088189 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-164c05b7-91f3-4f86-aa19-dae3c524ce0f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.285203 Oct 20 09:02:08.089004 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 129/235] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:07 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:08.209179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:08.209816 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: 8e67be6e-328c-4a4f-8a2b-083b26c3d62a] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:08.214863 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=81,nova_api:UPDATE=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:08.216619 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:08.217733 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:08.218204 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:08.218906 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:08.219527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:08.219952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:08.235279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] HTTP exception thrown: Instance 8e67be6e-328c-4a4f-8a2b-083b26c3d62a could not be found. Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Request method failure captured: Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a HTTP/1.1 Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ5qkfCJv3bhU1P64sw6kE5bR1OrRjcvm2iWVdCWnM7BxDdCmiN7RJHPdD8BF24tEsLwMHzLkQxLFGUGO2YfLi3Cw4793gFjP-nbgPyNrGnW5fsoGSmhc8sz8xwQuLNqn_B_Mio7_-f43SyWXa5UkAVfrSch1_VyQaNxl1WrrsboAQM Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 847f0b8423334fc7b505313a10845bcb Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-VolumesActionsTest-823875808 Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:02:08.240304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/847f0b8423334fc7b505313a10845bcb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_847f0b8423334fc7b505313a10845bcb", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/847f0b8423334fc7b505313a10845bcb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/847f0b8423334fc7b505313a10845bcb"}]}] Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-VolumesActionsTest-823875808 Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 847f0b8423334fc7b505313a10845bcb Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-VolumesActionsTest-823875808 Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-VolumesActionsTest-823875808-project Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 930afcdb977f4ed2a33b366563a755f1 Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-VolumesActionsTest-823875808-project Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 8e67be6e-328c-4a4f-8a2b-083b26c3d62a could not be found. Oct 20 09:02:08.242410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '8e67be6e-328c-4a4f-8a2b-083b26c3d62a'} Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8e67be6e-328c-4a4f-8a2b-083b26c3d62a could not be found. Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:02:08.243146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8e67be6e-328c-4a4f-8a2b-083b26c3d62a could not be found. Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8e67be6e-328c-4a4f-8a2b-083b26c3d62a could not be found. Oct 20 09:02:08.244279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:02:08.249415 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Returning 404 to user: Instance 8e67be6e-328c-4a4f-8a2b-083b26c3d62a could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:02:08.249415 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-16b59b3c-27c1-4a43-980e-277a0b2e5311 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a" status: 404 len: 111 microversion: 2.1 time: 0.038905 Oct 20 09:02:08.249415 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 107/236] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:08 2022] GET /compute/v2.1/servers/8e67be6e-328c-4a4f-8a2b-083b26c3d62a => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:02:08.364900 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1888374739", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "networks": [{"uuid": "4ee4717f-cbab-45d5-be89-db298134faad"}]}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:08.441632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] validate_networks() for [('4ee4717f-cbab-45d5-be89-db298134faad', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:02:09.008567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-32bb23a2-090a-44cd-84b6-bd284801cddf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:09.009813 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-32bb23a2-090a-44cd-84b6-bd284801cddf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:09.016012 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32bb23a2-090a-44cd-84b6-bd284801cddf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:09.016707 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32bb23a2-090a-44cd-84b6-bd284801cddf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:09.017320 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32bb23a2-090a-44cd-84b6-bd284801cddf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:09.018046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32bb23a2-090a-44cd-84b6-bd284801cddf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:09.018501 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32bb23a2-090a-44cd-84b6-bd284801cddf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:09.019092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32bb23a2-090a-44cd-84b6-bd284801cddf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:09.169174 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-32bb23a2-090a-44cd-84b6-bd284801cddf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:09.193887 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-32bb23a2-090a-44cd-84b6-bd284801cddf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.189869 Oct 20 09:02:09.194535 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 108/237] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:09 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:09.207208 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8a26fccd-04b9-4db5-916c-65105ee0946e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:09.208133 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8a26fccd-04b9-4db5-916c-65105ee0946e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:09.214702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a26fccd-04b9-4db5-916c-65105ee0946e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:09.215649 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a26fccd-04b9-4db5-916c-65105ee0946e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:09.216245 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a26fccd-04b9-4db5-916c-65105ee0946e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:09.217092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a26fccd-04b9-4db5-916c-65105ee0946e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:09.217707 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a26fccd-04b9-4db5-916c-65105ee0946e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:09.218392 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a26fccd-04b9-4db5-916c-65105ee0946e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:09.343462 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:02:09.344601 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:02:09.345278 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:02:09.345941 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:02:09.346767 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:02:09.361201 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:09.362130 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:09.363474 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:09.364309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:09.364998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:09.365665 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:09.391741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Getting quotas for project 847f0b8423334fc7b505313a10845bcb. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:02:09.396681 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Getting quotas for user 930afcdb977f4ed2a33b366563a755f1 and project 847f0b8423334fc7b505313a10845bcb. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:02:09.417316 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:02:09.420519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:02:09.457238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8a26fccd-04b9-4db5-916c-65105ee0946e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:09.466791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8a26fccd-04b9-4db5-916c-65105ee0946e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.263197 Oct 20 09:02:09.468286 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 109/238] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:09 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:09.473935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-704478973", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "26c5cd40-c195-42e5-baf1-db47fbf3d5b7"}], "security_groups": [{"name": "tempest-securitygroup--1816961322"}], "key_name": "tempest-keypair-944100248", "user_data": "}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:09.485922 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3d809f0d-7dff-44bf-b3db-39db70f57382 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.125542 Oct 20 09:02:09.493199 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 130/239] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:02:08 2022] POST /compute/v2.1/servers => generated 380 bytes in 1127 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:02:09.500523 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-019fed13-5f5c-4e24-9c5f-08b118a329e3 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:09.501780 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-019fed13-5f5c-4e24-9c5f-08b118a329e3 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:09.581763 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-019fed13-5f5c-4e24-9c5f-08b118a329e3 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:09.591296 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-019fed13-5f5c-4e24-9c5f-08b118a329e3 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1214 microversion: 2.1 time: 0.096608 Oct 20 09:02:09.593395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 131/240] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:09 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1214 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:09.671773 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] validate_networks() for [('26c5cd40-c195-42e5-baf1-db47fbf3d5b7', None, None, None, None, None)] {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:02:10.217804 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-286bf78e-f08e-4076-9824-a8dcb9e86ca8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:10.218742 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-286bf78e-f08e-4076-9824-a8dcb9e86ca8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:10.226316 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-286bf78e-f08e-4076-9824-a8dcb9e86ca8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:10.226967 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-286bf78e-f08e-4076-9824-a8dcb9e86ca8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:10.227486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-286bf78e-f08e-4076-9824-a8dcb9e86ca8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:10.228256 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-286bf78e-f08e-4076-9824-a8dcb9e86ca8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:10.228920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-286bf78e-f08e-4076-9824-a8dcb9e86ca8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:10.229416 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-286bf78e-f08e-4076-9824-a8dcb9e86ca8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:10.372384 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:02:10.373367 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:02:10.374056 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:02:10.374735 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:02:10.376041 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:02:10.389707 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:10.390542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:10.391240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:10.392191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:10.392870 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:10.393361 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:10.415455 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Getting quotas for project f1061324903c42ed9f0306471a113b82. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:02:10.419747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Getting quotas for user be1f82d7a0a84fbebdd0dffef9f098ea and project f1061324903c42ed9f0306471a113b82. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:02:10.433417 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:02:10.436557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:02:10.436815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-286bf78e-f08e-4076-9824-a8dcb9e86ca8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:10.445995 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-286bf78e-f08e-4076-9824-a8dcb9e86ca8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.232992 Oct 20 09:02:10.447201 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 132/241] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:10 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:10.490497 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-21338c01-280d-430e-afb6-f2a3930d450e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.020731 Oct 20 09:02:10.491731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 110/242] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:02:09 2022] POST /compute/v2.1/servers => generated 402 bytes in 1023 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:02:10.498179 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-32594223-cf2d-4106-9bbd-7f42de3470c7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:10.498821 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-32594223-cf2d-4106-9bbd-7f42de3470c7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:10.503064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8c7943a7-bf80-4e89-b15b-c8326e6c3e22 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:10.503672 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8c7943a7-bf80-4e89-b15b-c8326e6c3e22 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:10.507369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-32594223-cf2d-4106-9bbd-7f42de3470c7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:10.508084 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-32594223-cf2d-4106-9bbd-7f42de3470c7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:10.508684 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-32594223-cf2d-4106-9bbd-7f42de3470c7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:10.509585 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-32594223-cf2d-4106-9bbd-7f42de3470c7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:10.510240 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-32594223-cf2d-4106-9bbd-7f42de3470c7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:10.510879 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-32594223-cf2d-4106-9bbd-7f42de3470c7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:10.600841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8c7943a7-bf80-4e89-b15b-c8326e6c3e22 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:10.613344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8c7943a7-bf80-4e89-b15b-c8326e6c3e22 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1236 microversion: 2.1 time: 0.114415 Oct 20 09:02:10.613932 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 111/243] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:10 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1236 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:10.644029 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-30935ac6-5b1f-48b8-abd1-4e8dd5b5c448 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:10.644974 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-30935ac6-5b1f-48b8-abd1-4e8dd5b5c448 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:10.664265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30935ac6-5b1f-48b8-abd1-4e8dd5b5c448 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:10.665104 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30935ac6-5b1f-48b8-abd1-4e8dd5b5c448 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:10.665814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30935ac6-5b1f-48b8-abd1-4e8dd5b5c448 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:10.667114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30935ac6-5b1f-48b8-abd1-4e8dd5b5c448 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:10.667852 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30935ac6-5b1f-48b8-abd1-4e8dd5b5c448 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:10.668591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30935ac6-5b1f-48b8-abd1-4e8dd5b5c448 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:10.746060 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-30935ac6-5b1f-48b8-abd1-4e8dd5b5c448 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:10.765175 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-30935ac6-5b1f-48b8-abd1-4e8dd5b5c448 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1266 microversion: 2.1 time: 0.133235 Oct 20 09:02:10.766407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 112/244] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:10 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1266 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:10.897790 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-32594223-cf2d-4106-9bbd-7f42de3470c7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:10.910658 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-32594223-cf2d-4106-9bbd-7f42de3470c7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.420850 Oct 20 09:02:10.911665 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 133/245] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:10 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:11.469041 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c72f8d6c-4a75-46c2-b471-9afc393a2dd2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:11.469672 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c72f8d6c-4a75-46c2-b471-9afc393a2dd2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:11.476191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c72f8d6c-4a75-46c2-b471-9afc393a2dd2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:11.476709 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c72f8d6c-4a75-46c2-b471-9afc393a2dd2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:11.477139 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c72f8d6c-4a75-46c2-b471-9afc393a2dd2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:11.477810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c72f8d6c-4a75-46c2-b471-9afc393a2dd2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:11.478256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c72f8d6c-4a75-46c2-b471-9afc393a2dd2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:11.478703 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c72f8d6c-4a75-46c2-b471-9afc393a2dd2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:11.608409 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c72f8d6c-4a75-46c2-b471-9afc393a2dd2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:11.615965 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c72f8d6c-4a75-46c2-b471-9afc393a2dd2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.150918 Oct 20 09:02:11.616462 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 113/246] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:11 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:11.637072 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a06d868f-4c3e-435a-80c8-e0be6a475dfb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:11.637772 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a06d868f-4c3e-435a-80c8-e0be6a475dfb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:11.644410 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a06d868f-4c3e-435a-80c8-e0be6a475dfb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:11.644889 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a06d868f-4c3e-435a-80c8-e0be6a475dfb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:11.645322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a06d868f-4c3e-435a-80c8-e0be6a475dfb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:11.646002 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a06d868f-4c3e-435a-80c8-e0be6a475dfb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:11.646451 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a06d868f-4c3e-435a-80c8-e0be6a475dfb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:11.646890 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a06d868f-4c3e-435a-80c8-e0be6a475dfb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:11.706806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a06d868f-4c3e-435a-80c8-e0be6a475dfb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:11.715549 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a06d868f-4c3e-435a-80c8-e0be6a475dfb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1240 microversion: 2.1 time: 0.082073 Oct 20 09:02:11.716276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 134/247] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:11 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1240 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:11.789471 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ceaec26f-f640-4cab-804b-498da54874dc tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:11.790181 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ceaec26f-f640-4cab-804b-498da54874dc tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:11.796150 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ceaec26f-f640-4cab-804b-498da54874dc tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:11.796730 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ceaec26f-f640-4cab-804b-498da54874dc tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:11.797238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ceaec26f-f640-4cab-804b-498da54874dc tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:11.797928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ceaec26f-f640-4cab-804b-498da54874dc tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:11.798376 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ceaec26f-f640-4cab-804b-498da54874dc tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:11.798826 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ceaec26f-f640-4cab-804b-498da54874dc tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:11.859471 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ceaec26f-f640-4cab-804b-498da54874dc tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:11.867201 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ceaec26f-f640-4cab-804b-498da54874dc tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1266 microversion: 2.1 time: 0.082204 Oct 20 09:02:11.867710 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 114/248] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:11 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1266 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:11.937189 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-cedbd3fc-ae71-44b8-8175-37ae89c9bcff tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:11.937825 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-cedbd3fc-ae71-44b8-8175-37ae89c9bcff tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:11.943727 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cedbd3fc-ae71-44b8-8175-37ae89c9bcff tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:11.944759 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cedbd3fc-ae71-44b8-8175-37ae89c9bcff tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:11.945196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cedbd3fc-ae71-44b8-8175-37ae89c9bcff tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:11.945861 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cedbd3fc-ae71-44b8-8175-37ae89c9bcff tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:11.946326 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cedbd3fc-ae71-44b8-8175-37ae89c9bcff tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:11.946761 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cedbd3fc-ae71-44b8-8175-37ae89c9bcff tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:12.074888 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-cedbd3fc-ae71-44b8-8175-37ae89c9bcff tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:12.082478 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-cedbd3fc-ae71-44b8-8175-37ae89c9bcff tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.149218 Oct 20 09:02:12.083053 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 135/249] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:11 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:12.638740 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d7759743-4f97-4dca-993b-fdba0f96600e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:12.639349 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d7759743-4f97-4dca-993b-fdba0f96600e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:12.645277 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7759743-4f97-4dca-993b-fdba0f96600e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:12.645758 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7759743-4f97-4dca-993b-fdba0f96600e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:12.646201 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7759743-4f97-4dca-993b-fdba0f96600e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:12.646910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7759743-4f97-4dca-993b-fdba0f96600e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:12.647365 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7759743-4f97-4dca-993b-fdba0f96600e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:12.647784 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d7759743-4f97-4dca-993b-fdba0f96600e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:12.739093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-173ffcc1-76c1-4a8a-8b7b-fc90f13a29ce tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:12.739692 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-173ffcc1-76c1-4a8a-8b7b-fc90f13a29ce tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:12.746659 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-173ffcc1-76c1-4a8a-8b7b-fc90f13a29ce tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:12.747156 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-173ffcc1-76c1-4a8a-8b7b-fc90f13a29ce tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:12.747587 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-173ffcc1-76c1-4a8a-8b7b-fc90f13a29ce tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:12.748276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-173ffcc1-76c1-4a8a-8b7b-fc90f13a29ce tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:12.748717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-173ffcc1-76c1-4a8a-8b7b-fc90f13a29ce tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:12.749134 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-173ffcc1-76c1-4a8a-8b7b-fc90f13a29ce tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:12.776699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d7759743-4f97-4dca-993b-fdba0f96600e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:12.784347 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d7759743-4f97-4dca-993b-fdba0f96600e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.149528 Oct 20 09:02:12.784913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 115/250] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:12 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:12.808157 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-173ffcc1-76c1-4a8a-8b7b-fc90f13a29ce tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:12.815948 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-173ffcc1-76c1-4a8a-8b7b-fc90f13a29ce tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1240 microversion: 2.1 time: 0.080754 Oct 20 09:02:12.816499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 136/251] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:12 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1240 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:12.889995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-577ddaec-e74a-45b4-9348-4b16443ea583 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:12.890554 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-577ddaec-e74a-45b4-9348-4b16443ea583 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:12.896439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-577ddaec-e74a-45b4-9348-4b16443ea583 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:12.896920 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-577ddaec-e74a-45b4-9348-4b16443ea583 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:12.897974 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-577ddaec-e74a-45b4-9348-4b16443ea583 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:12.898691 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-577ddaec-e74a-45b4-9348-4b16443ea583 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:12.899147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-577ddaec-e74a-45b4-9348-4b16443ea583 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:12.899573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-577ddaec-e74a-45b4-9348-4b16443ea583 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:12.958007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-577ddaec-e74a-45b4-9348-4b16443ea583 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:12.966484 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-577ddaec-e74a-45b4-9348-4b16443ea583 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1266 microversion: 2.1 time: 0.080402 Oct 20 09:02:12.967064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 116/252] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:12 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1266 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:13.105306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9725eb6e-7422-4328-b9ba-02edf3c39fa4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:13.105995 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9725eb6e-7422-4328-b9ba-02edf3c39fa4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:13.112010 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9725eb6e-7422-4328-b9ba-02edf3c39fa4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:13.112568 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9725eb6e-7422-4328-b9ba-02edf3c39fa4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:13.113095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9725eb6e-7422-4328-b9ba-02edf3c39fa4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:13.113854 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9725eb6e-7422-4328-b9ba-02edf3c39fa4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:13.114375 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9725eb6e-7422-4328-b9ba-02edf3c39fa4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:13.114887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9725eb6e-7422-4328-b9ba-02edf3c39fa4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:13.244304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9725eb6e-7422-4328-b9ba-02edf3c39fa4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:13.252131 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9725eb6e-7422-4328-b9ba-02edf3c39fa4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.150667 Oct 20 09:02:13.252706 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 137/253] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:13 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:13.806850 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1b1dda31-4f7c-4fa9-ada1-fc30b75b5e0a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:13.807473 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1b1dda31-4f7c-4fa9-ada1-fc30b75b5e0a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:13.813427 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1b1dda31-4f7c-4fa9-ada1-fc30b75b5e0a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:13.813895 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1b1dda31-4f7c-4fa9-ada1-fc30b75b5e0a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:13.814323 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1b1dda31-4f7c-4fa9-ada1-fc30b75b5e0a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:13.815051 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1b1dda31-4f7c-4fa9-ada1-fc30b75b5e0a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:13.815496 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1b1dda31-4f7c-4fa9-ada1-fc30b75b5e0a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:13.815908 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1b1dda31-4f7c-4fa9-ada1-fc30b75b5e0a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:13.830394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:13.838807 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6bd40f11-80bb-4981-8139-ba19aa2860f4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:13.839937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6bd40f11-80bb-4981-8139-ba19aa2860f4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:13.846597 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bd40f11-80bb-4981-8139-ba19aa2860f4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:13.847082 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bd40f11-80bb-4981-8139-ba19aa2860f4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:13.847517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bd40f11-80bb-4981-8139-ba19aa2860f4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:13.848209 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bd40f11-80bb-4981-8139-ba19aa2860f4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:13.848666 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bd40f11-80bb-4981-8139-ba19aa2860f4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:13.849088 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bd40f11-80bb-4981-8139-ba19aa2860f4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:13.912596 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6bd40f11-80bb-4981-8139-ba19aa2860f4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:13.920705 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6bd40f11-80bb-4981-8139-ba19aa2860f4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1240 microversion: 2.1 time: 0.086287 Oct 20 09:02:13.921267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 138/254] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:13 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1240 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:13.943258 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1b1dda31-4f7c-4fa9-ada1-fc30b75b5e0a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:13.950995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1b1dda31-4f7c-4fa9-ada1-fc30b75b5e0a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1520 microversion: 2.1 time: 0.147982 Oct 20 09:02:13.951544 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 117/255] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:13 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1520 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:13.988925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5f654839-e593-413f-831f-4835b9b5caee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:13.989450 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5f654839-e593-413f-831f-4835b9b5caee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:13.995349 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f654839-e593-413f-831f-4835b9b5caee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:13.995825 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f654839-e593-413f-831f-4835b9b5caee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:13.996254 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f654839-e593-413f-831f-4835b9b5caee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:13.996920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f654839-e593-413f-831f-4835b9b5caee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:13.997362 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f654839-e593-413f-831f-4835b9b5caee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:13.997811 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f654839-e593-413f-831f-4835b9b5caee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:14.065875 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5f654839-e593-413f-831f-4835b9b5caee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:14.073801 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5f654839-e593-413f-831f-4835b9b5caee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1266 microversion: 2.1 time: 0.088793 Oct 20 09:02:14.074961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 139/256] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:13 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1266 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:14.276285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1ae97830-1569-4eb2-91db-209808d61706 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:14.276818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1ae97830-1569-4eb2-91db-209808d61706 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:14.282796 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ae97830-1569-4eb2-91db-209808d61706 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:14.283284 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ae97830-1569-4eb2-91db-209808d61706 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:14.283723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ae97830-1569-4eb2-91db-209808d61706 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:14.284407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ae97830-1569-4eb2-91db-209808d61706 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:14.284865 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ae97830-1569-4eb2-91db-209808d61706 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:14.285290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ae97830-1569-4eb2-91db-209808d61706 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:14.415933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1ae97830-1569-4eb2-91db-209808d61706 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:14.423505 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1ae97830-1569-4eb2-91db-209808d61706 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.151735 Oct 20 09:02:14.424060 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 118/257] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:14 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:14.943550 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4a389f72-91e2-42ee-bb5b-61b1b896eb34 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:14.944728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4a389f72-91e2-42ee-bb5b-61b1b896eb34 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:14.951125 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a389f72-91e2-42ee-bb5b-61b1b896eb34 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:14.951605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a389f72-91e2-42ee-bb5b-61b1b896eb34 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:14.952040 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a389f72-91e2-42ee-bb5b-61b1b896eb34 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:14.952718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a389f72-91e2-42ee-bb5b-61b1b896eb34 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:14.953170 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a389f72-91e2-42ee-bb5b-61b1b896eb34 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:14.953596 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4a389f72-91e2-42ee-bb5b-61b1b896eb34 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:14.974343 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b4dd66c4-64ca-4ac3-9ee5-f2e920d79b44 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:14.975108 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b4dd66c4-64ca-4ac3-9ee5-f2e920d79b44 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:14.981017 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4dd66c4-64ca-4ac3-9ee5-f2e920d79b44 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:14.981492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4dd66c4-64ca-4ac3-9ee5-f2e920d79b44 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:14.981928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4dd66c4-64ca-4ac3-9ee5-f2e920d79b44 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:14.982645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4dd66c4-64ca-4ac3-9ee5-f2e920d79b44 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:14.983095 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4dd66c4-64ca-4ac3-9ee5-f2e920d79b44 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:14.983520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4dd66c4-64ca-4ac3-9ee5-f2e920d79b44 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:15.015107 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4a389f72-91e2-42ee-bb5b-61b1b896eb34 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:15.023385 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4a389f72-91e2-42ee-bb5b-61b1b896eb34 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1288 microversion: 2.1 time: 0.083733 Oct 20 09:02:15.024160 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 140/258] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:14 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1288 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:15.096648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-344fe52c-8744-4d3c-a9de-9d46bd8cd55e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:15.097384 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-344fe52c-8744-4d3c-a9de-9d46bd8cd55e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:15.103263 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-344fe52c-8744-4d3c-a9de-9d46bd8cd55e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:15.103745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-344fe52c-8744-4d3c-a9de-9d46bd8cd55e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:15.104181 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-344fe52c-8744-4d3c-a9de-9d46bd8cd55e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:15.104875 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-344fe52c-8744-4d3c-a9de-9d46bd8cd55e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:15.105446 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-344fe52c-8744-4d3c-a9de-9d46bd8cd55e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:15.105877 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-344fe52c-8744-4d3c-a9de-9d46bd8cd55e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:15.111743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b4dd66c4-64ca-4ac3-9ee5-f2e920d79b44 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:15.119482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b4dd66c4-64ca-4ac3-9ee5-f2e920d79b44 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1500 microversion: 2.1 time: 0.149002 Oct 20 09:02:15.128452 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 119/259] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:14 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1500 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:15.141482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-980cfbe7-635d-4269-adff-d72979f9cf0b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:15.142048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-980cfbe7-635d-4269-adff-d72979f9cf0b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:15.153285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-980cfbe7-635d-4269-adff-d72979f9cf0b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:15.153783 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-980cfbe7-635d-4269-adff-d72979f9cf0b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:15.154224 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-980cfbe7-635d-4269-adff-d72979f9cf0b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:15.154958 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-980cfbe7-635d-4269-adff-d72979f9cf0b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:15.155415 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-980cfbe7-635d-4269-adff-d72979f9cf0b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:15.155841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-980cfbe7-635d-4269-adff-d72979f9cf0b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:15.170492 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-344fe52c-8744-4d3c-a9de-9d46bd8cd55e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:15.179778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-344fe52c-8744-4d3c-a9de-9d46bd8cd55e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1274 microversion: 2.1 time: 0.086899 Oct 20 09:02:15.180586 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 141/260] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:15 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1274 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:15.290508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-980cfbe7-635d-4269-adff-d72979f9cf0b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:15.298019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-980cfbe7-635d-4269-adff-d72979f9cf0b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1514 microversion: 2.1 time: 0.160341 Oct 20 09:02:15.298606 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 120/261] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:15 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1514 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:15.454022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9c0b928e-74d7-42f1-9c61-1647cb4c1384 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:15.454763 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9c0b928e-74d7-42f1-9c61-1647cb4c1384 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:15.464256 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c0b928e-74d7-42f1-9c61-1647cb4c1384 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:15.464775 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c0b928e-74d7-42f1-9c61-1647cb4c1384 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:15.465468 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c0b928e-74d7-42f1-9c61-1647cb4c1384 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:15.470542 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c0b928e-74d7-42f1-9c61-1647cb4c1384 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:15.471232 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c0b928e-74d7-42f1-9c61-1647cb4c1384 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:15.472647 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c0b928e-74d7-42f1-9c61-1647cb4c1384 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:15.764629 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9c0b928e-74d7-42f1-9c61-1647cb4c1384 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:15.772994 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9c0b928e-74d7-42f1-9c61-1647cb4c1384 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.330279 Oct 20 09:02:15.773857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 142/262] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:15 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:16.047285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-96ec9fad-ad10-4e67-b5e4-82e1168330cf tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:16.047899 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-96ec9fad-ad10-4e67-b5e4-82e1168330cf tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:16.056244 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-96ec9fad-ad10-4e67-b5e4-82e1168330cf tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:16.056873 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-96ec9fad-ad10-4e67-b5e4-82e1168330cf tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:16.057383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-96ec9fad-ad10-4e67-b5e4-82e1168330cf tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:16.058277 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-96ec9fad-ad10-4e67-b5e4-82e1168330cf tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:16.058757 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-96ec9fad-ad10-4e67-b5e4-82e1168330cf tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:16.059192 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-96ec9fad-ad10-4e67-b5e4-82e1168330cf tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:16.152310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-311a118e-8dce-4a7c-a7b9-8f2b3bde2d51 req-2246a723-076f-4be2-84f7-43023dfc984d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fe511cdb-e20c-4ec9-a37b-1eb239d3ed67", "name": "network-vif-plugged", "status": "completed", "tag": "6bd83ee3-a5b5-413b-9b18-8034518260a1"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:16.163356 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-311a118e-8dce-4a7c-a7b9-8f2b3bde2d51 req-2246a723-076f-4be2-84f7-43023dfc984d service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:16.164002 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-311a118e-8dce-4a7c-a7b9-8f2b3bde2d51 req-2246a723-076f-4be2-84f7-43023dfc984d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:16.164567 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-311a118e-8dce-4a7c-a7b9-8f2b3bde2d51 req-2246a723-076f-4be2-84f7-43023dfc984d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:16.167761 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-96ec9fad-ad10-4e67-b5e4-82e1168330cf tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:16.179461 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-311a118e-8dce-4a7c-a7b9-8f2b3bde2d51 req-2246a723-076f-4be2-84f7-43023dfc984d service nova] Creating event network-vif-plugged:6bd83ee3-a5b5-413b-9b18-8034518260a1 for instance fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 on hv-cinder-maste Oct 20 09:02:16.185781 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-311a118e-8dce-4a7c-a7b9-8f2b3bde2d51 req-2246a723-076f-4be2-84f7-43023dfc984d service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037183 Oct 20 09:02:16.186592 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 143/263] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:02:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:02:16.189790 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-96ec9fad-ad10-4e67-b5e4-82e1168330cf tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1288 microversion: 2.1 time: 0.146173 Oct 20 09:02:16.193791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 121/264] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:16 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1288 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:16.216757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2f9c573f-9783-43a5-99ac-98f8ef8eb21c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:16.220138 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2f9c573f-9783-43a5-99ac-98f8ef8eb21c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:16.227341 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2f9c573f-9783-43a5-99ac-98f8ef8eb21c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:16.227886 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2f9c573f-9783-43a5-99ac-98f8ef8eb21c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:16.228437 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2f9c573f-9783-43a5-99ac-98f8ef8eb21c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:16.229380 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2f9c573f-9783-43a5-99ac-98f8ef8eb21c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:16.229911 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2f9c573f-9783-43a5-99ac-98f8ef8eb21c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:16.230336 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2f9c573f-9783-43a5-99ac-98f8ef8eb21c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:16.380032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2f9c573f-9783-43a5-99ac-98f8ef8eb21c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:16.390007 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2f9c573f-9783-43a5-99ac-98f8ef8eb21c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.188562 Oct 20 09:02:16.390836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 144/265] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:16 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:16.800622 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c2da5cc2-7ff4-4858-aa3c-d543ad585f40 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:16.801903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c2da5cc2-7ff4-4858-aa3c-d543ad585f40 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:16.809952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2da5cc2-7ff4-4858-aa3c-d543ad585f40 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:16.810893 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2da5cc2-7ff4-4858-aa3c-d543ad585f40 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:16.811693 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2da5cc2-7ff4-4858-aa3c-d543ad585f40 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:16.812616 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2da5cc2-7ff4-4858-aa3c-d543ad585f40 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:16.814649 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2da5cc2-7ff4-4858-aa3c-d543ad585f40 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:16.815126 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2da5cc2-7ff4-4858-aa3c-d543ad585f40 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:16.866737 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-42e0e731-46a5-44f7-8291-66e2ee0db524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:16.869150 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-42e0e731-46a5-44f7-8291-66e2ee0db524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:16.875247 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42e0e731-46a5-44f7-8291-66e2ee0db524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:16.875807 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42e0e731-46a5-44f7-8291-66e2ee0db524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:16.876318 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42e0e731-46a5-44f7-8291-66e2ee0db524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:16.877093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42e0e731-46a5-44f7-8291-66e2ee0db524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:16.877817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42e0e731-46a5-44f7-8291-66e2ee0db524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:16.878309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-42e0e731-46a5-44f7-8291-66e2ee0db524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:16.905491 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-42e0e731-46a5-44f7-8291-66e2ee0db524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lazy-loading 'flavor' on Instance uuid 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:02:16.977209 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:17.020263 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-42e0e731-46a5-44f7-8291-66e2ee0db524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150/action" status: 200 len: 32 microversion: 2.1 time: 0.158012 Oct 20 09:02:17.020948 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 145/266] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:02:16 2022] POST /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150/action => generated 32 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 20 09:02:17.030313 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a751c5a1-2b5d-4965-8398-de5bf6e27caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:17.031026 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a751c5a1-2b5d-4965-8398-de5bf6e27caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:17.039665 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a751c5a1-2b5d-4965-8398-de5bf6e27caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:17.040143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a751c5a1-2b5d-4965-8398-de5bf6e27caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:17.040567 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a751c5a1-2b5d-4965-8398-de5bf6e27caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:17.041249 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a751c5a1-2b5d-4965-8398-de5bf6e27caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:17.041688 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a751c5a1-2b5d-4965-8398-de5bf6e27caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:17.042101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a751c5a1-2b5d-4965-8398-de5bf6e27caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:17.048433 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c2da5cc2-7ff4-4858-aa3c-d543ad585f40 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:17.056884 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c2da5cc2-7ff4-4858-aa3c-d543ad585f40 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1319 microversion: 2.1 time: 0.263588 Oct 20 09:02:17.063776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 122/267] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:16 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1319 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:17.185052 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a751c5a1-2b5d-4965-8398-de5bf6e27caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:17.193642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a751c5a1-2b5d-4965-8398-de5bf6e27caf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.167179 Oct 20 09:02:17.194857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 146/268] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:17 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:17.215723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f57f3077-1d1a-4b6b-acc0-0f4c423721d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:17.216335 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f57f3077-1d1a-4b6b-acc0-0f4c423721d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:17.222786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f57f3077-1d1a-4b6b-acc0-0f4c423721d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:17.223337 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f57f3077-1d1a-4b6b-acc0-0f4c423721d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:17.223775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f57f3077-1d1a-4b6b-acc0-0f4c423721d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:17.224517 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f57f3077-1d1a-4b6b-acc0-0f4c423721d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:17.224994 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f57f3077-1d1a-4b6b-acc0-0f4c423721d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:17.226040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f57f3077-1d1a-4b6b-acc0-0f4c423721d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:17.281489 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-2e3a8bdf-597f-498b-b6fd-04eeb258a1a5 req-47c39c6c-7503-41e2-ac0f-e03b8c0dd4cd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "df3974e9-9d16-4e6b-ac63-0f1257ff6d7c", "tag": "733a3a52-e016-43f3-9379-3f4b29beac85"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:17.291282 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-2e3a8bdf-597f-498b-b6fd-04eeb258a1a5 req-47c39c6c-7503-41e2-ac0f-e03b8c0dd4cd service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:17.291759 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-2e3a8bdf-597f-498b-b6fd-04eeb258a1a5 req-47c39c6c-7503-41e2-ac0f-e03b8c0dd4cd service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:17.292194 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-2e3a8bdf-597f-498b-b6fd-04eeb258a1a5 req-47c39c6c-7503-41e2-ac0f-e03b8c0dd4cd service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:17.304664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-2e3a8bdf-597f-498b-b6fd-04eeb258a1a5 req-47c39c6c-7503-41e2-ac0f-e03b8c0dd4cd service nova] Creating event network-changed:733a3a52-e016-43f3-9379-3f4b29beac85 for instance df3974e9-9d16-4e6b-ac63-0f1257ff6d7c on hv-cinder-maste Oct 20 09:02:17.309509 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-2e3a8bdf-597f-498b-b6fd-04eeb258a1a5 req-47c39c6c-7503-41e2-ac0f-e03b8c0dd4cd service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031663 Oct 20 09:02:17.310263 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 147/269] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:02:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:02:17.322933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f57f3077-1d1a-4b6b-acc0-0f4c423721d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:17.330769 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f57f3077-1d1a-4b6b-acc0-0f4c423721d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1294 microversion: 2.1 time: 0.119188 Oct 20 09:02:17.331340 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 123/270] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:17 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1294 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:17.414883 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8207015f-7760-4e60-b890-5338d0c81674 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:17.415564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8207015f-7760-4e60-b890-5338d0c81674 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:17.422300 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8207015f-7760-4e60-b890-5338d0c81674 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:17.422886 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8207015f-7760-4e60-b890-5338d0c81674 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:17.423394 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8207015f-7760-4e60-b890-5338d0c81674 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:17.424155 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8207015f-7760-4e60-b890-5338d0c81674 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:17.424636 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8207015f-7760-4e60-b890-5338d0c81674 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:17.425058 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8207015f-7760-4e60-b890-5338d0c81674 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:17.559776 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8207015f-7760-4e60-b890-5338d0c81674 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:17.567741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8207015f-7760-4e60-b890-5338d0c81674 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.157797 Oct 20 09:02:17.568306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 148/271] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:17 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:18.080712 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5bdc658e-39e2-4409-b4d9-a9da937bbadc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:18.081313 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5bdc658e-39e2-4409-b4d9-a9da937bbadc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:18.088126 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bdc658e-39e2-4409-b4d9-a9da937bbadc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:18.088793 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bdc658e-39e2-4409-b4d9-a9da937bbadc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:18.089407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bdc658e-39e2-4409-b4d9-a9da937bbadc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:18.090587 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bdc658e-39e2-4409-b4d9-a9da937bbadc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:18.091405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bdc658e-39e2-4409-b4d9-a9da937bbadc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:18.091837 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bdc658e-39e2-4409-b4d9-a9da937bbadc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:18.217516 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1af18886-3cb5-4323-9544-e88524f0f3f0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:18.218212 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1af18886-3cb5-4323-9544-e88524f0f3f0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:18.224567 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1af18886-3cb5-4323-9544-e88524f0f3f0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:18.225117 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1af18886-3cb5-4323-9544-e88524f0f3f0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:18.225618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1af18886-3cb5-4323-9544-e88524f0f3f0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:18.227015 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1af18886-3cb5-4323-9544-e88524f0f3f0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:18.227464 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1af18886-3cb5-4323-9544-e88524f0f3f0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:18.227882 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1af18886-3cb5-4323-9544-e88524f0f3f0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:18.295294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5bdc658e-39e2-4409-b4d9-a9da937bbadc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:18.304455 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5bdc658e-39e2-4409-b4d9-a9da937bbadc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1507 microversion: 2.1 time: 0.227568 Oct 20 09:02:18.305656 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 124/272] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:18 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1507 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:18.356253 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-87d1f0bf-23c7-4670-acd8-28b25ff0dceb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:18.357022 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-87d1f0bf-23c7-4670-acd8-28b25ff0dceb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:18.372067 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-87d1f0bf-23c7-4670-acd8-28b25ff0dceb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:18.372564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-87d1f0bf-23c7-4670-acd8-28b25ff0dceb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:18.373170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-87d1f0bf-23c7-4670-acd8-28b25ff0dceb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:18.374047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-87d1f0bf-23c7-4670-acd8-28b25ff0dceb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:18.374708 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-87d1f0bf-23c7-4670-acd8-28b25ff0dceb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:18.375129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-87d1f0bf-23c7-4670-acd8-28b25ff0dceb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:18.390149 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1af18886-3cb5-4323-9544-e88524f0f3f0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:18.398693 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1af18886-3cb5-4323-9544-e88524f0f3f0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.185215 Oct 20 09:02:18.418808 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 149/273] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:18 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:18.467628 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-87d1f0bf-23c7-4670-acd8-28b25ff0dceb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:18.477227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-87d1f0bf-23c7-4670-acd8-28b25ff0dceb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1294 microversion: 2.1 time: 0.124960 Oct 20 09:02:18.477802 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 125/274] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:18 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1294 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:18.590741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-62833322-5313-46ce-9d0b-3c5a8e3ee914 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:18.591796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-62833322-5313-46ce-9d0b-3c5a8e3ee914 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:18.604342 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62833322-5313-46ce-9d0b-3c5a8e3ee914 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:18.605126 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62833322-5313-46ce-9d0b-3c5a8e3ee914 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:18.607556 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62833322-5313-46ce-9d0b-3c5a8e3ee914 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:18.608334 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62833322-5313-46ce-9d0b-3c5a8e3ee914 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:18.608880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62833322-5313-46ce-9d0b-3c5a8e3ee914 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:18.610122 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62833322-5313-46ce-9d0b-3c5a8e3ee914 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:18.885414 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-62833322-5313-46ce-9d0b-3c5a8e3ee914 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:18.914509 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-62833322-5313-46ce-9d0b-3c5a8e3ee914 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.312836 Oct 20 09:02:18.914509 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 150/275] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:18 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:19.423595 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c4883ff2-58ce-410e-80df-6d6238c84e5b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:19.424776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c4883ff2-58ce-410e-80df-6d6238c84e5b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:19.439057 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c4883ff2-58ce-410e-80df-6d6238c84e5b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:19.439554 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c4883ff2-58ce-410e-80df-6d6238c84e5b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:19.439990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c4883ff2-58ce-410e-80df-6d6238c84e5b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:19.440695 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c4883ff2-58ce-410e-80df-6d6238c84e5b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:19.441771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c4883ff2-58ce-410e-80df-6d6238c84e5b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:19.442287 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c4883ff2-58ce-410e-80df-6d6238c84e5b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:19.501845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-46c70877-c1cf-4abb-bd75-ff5e5bc31446 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:19.502336 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-46c70877-c1cf-4abb-bd75-ff5e5bc31446 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:19.504853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:19.521100 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46c70877-c1cf-4abb-bd75-ff5e5bc31446 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:19.521512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46c70877-c1cf-4abb-bd75-ff5e5bc31446 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:19.521949 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46c70877-c1cf-4abb-bd75-ff5e5bc31446 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:19.524784 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46c70877-c1cf-4abb-bd75-ff5e5bc31446 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:19.525411 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46c70877-c1cf-4abb-bd75-ff5e5bc31446 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:19.527055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46c70877-c1cf-4abb-bd75-ff5e5bc31446 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:19.589745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=12 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:19.708858 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c4883ff2-58ce-410e-80df-6d6238c84e5b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:19.723804 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c4883ff2-58ce-410e-80df-6d6238c84e5b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.305035 Oct 20 09:02:19.740301 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 126/276] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:19 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:19.774015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-3973e30e-f7eb-4fc9-94da-141393c87bfd req-e6522698-1b70-40a4-be59-f0cd3544db6c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "26ae0069-be67-4921-aca3-4cfae5214cf5", "tag": "ead535c7-97fc-458e-b3a8-642e6ca66f0a"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:19.785453 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-3973e30e-f7eb-4fc9-94da-141393c87bfd req-e6522698-1b70-40a4-be59-f0cd3544db6c service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:19.786099 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-3973e30e-f7eb-4fc9-94da-141393c87bfd req-e6522698-1b70-40a4-be59-f0cd3544db6c service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:19.787662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-3973e30e-f7eb-4fc9-94da-141393c87bfd req-e6522698-1b70-40a4-be59-f0cd3544db6c service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:19.797105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-46c70877-c1cf-4abb-bd75-ff5e5bc31446 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:19.804347 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-3973e30e-f7eb-4fc9-94da-141393c87bfd req-e6522698-1b70-40a4-be59-f0cd3544db6c service nova] Creating event network-changed:ead535c7-97fc-458e-b3a8-642e6ca66f0a for instance 26ae0069-be67-4921-aca3-4cfae5214cf5 on hv-cinder-maste Oct 20 09:02:19.807915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-46c70877-c1cf-4abb-bd75-ff5e5bc31446 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.310559 Oct 20 09:02:19.811158 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-3973e30e-f7eb-4fc9-94da-141393c87bfd req-e6522698-1b70-40a4-be59-f0cd3544db6c service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041809 Oct 20 09:02:19.811941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 127/277] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:02:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:02:19.824815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 151/278] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:19 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:19.927548 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-50ba956a-5281-410f-8a78-9006c3e6d3af tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:19.928274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-50ba956a-5281-410f-8a78-9006c3e6d3af tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:19.935943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50ba956a-5281-410f-8a78-9006c3e6d3af tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:19.936433 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50ba956a-5281-410f-8a78-9006c3e6d3af tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:19.936872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50ba956a-5281-410f-8a78-9006c3e6d3af tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:19.937565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50ba956a-5281-410f-8a78-9006c3e6d3af tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:19.938048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50ba956a-5281-410f-8a78-9006c3e6d3af tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:19.938566 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50ba956a-5281-410f-8a78-9006c3e6d3af tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:20.142508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-50ba956a-5281-410f-8a78-9006c3e6d3af tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:20.152161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-50ba956a-5281-410f-8a78-9006c3e6d3af tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.228966 Oct 20 09:02:20.153286 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 128/279] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:19 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:20.747396 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-383836ca-a880-4c18-9901-711dae08f709 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:20.748001 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-383836ca-a880-4c18-9901-711dae08f709 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:20.750359 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:20.756607 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-383836ca-a880-4c18-9901-711dae08f709 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:20.757875 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-383836ca-a880-4c18-9901-711dae08f709 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:20.758323 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-383836ca-a880-4c18-9901-711dae08f709 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:20.759080 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-383836ca-a880-4c18-9901-711dae08f709 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:20.759555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-383836ca-a880-4c18-9901-711dae08f709 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:20.759979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-383836ca-a880-4c18-9901-711dae08f709 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:20.832144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4fc88b1f-8081-42d5-ab11-97c360a2a151 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:20.832887 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4fc88b1f-8081-42d5-ab11-97c360a2a151 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:20.834914 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:20.835723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=8 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:20.842007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4fc88b1f-8081-42d5-ab11-97c360a2a151 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:20.842502 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4fc88b1f-8081-42d5-ab11-97c360a2a151 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:20.842948 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4fc88b1f-8081-42d5-ab11-97c360a2a151 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:20.843807 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4fc88b1f-8081-42d5-ab11-97c360a2a151 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:20.844063 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4fc88b1f-8081-42d5-ab11-97c360a2a151 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:20.844479 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4fc88b1f-8081-42d5-ab11-97c360a2a151 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:20.898811 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-383836ca-a880-4c18-9901-711dae08f709 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:20.906491 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-383836ca-a880-4c18-9901-711dae08f709 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.163332 Oct 20 09:02:20.907057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 152/280] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:20 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:20.984709 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4fc88b1f-8081-42d5-ab11-97c360a2a151 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:20.992934 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4fc88b1f-8081-42d5-ab11-97c360a2a151 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.164577 Oct 20 09:02:20.993521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 129/281] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:20 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:21.177514 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-11103299-09da-4fe3-a6dc-ef85176c9ec8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:21.178155 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-11103299-09da-4fe3-a6dc-ef85176c9ec8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:21.184692 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11103299-09da-4fe3-a6dc-ef85176c9ec8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:21.185170 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11103299-09da-4fe3-a6dc-ef85176c9ec8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:21.185601 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11103299-09da-4fe3-a6dc-ef85176c9ec8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:21.186283 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11103299-09da-4fe3-a6dc-ef85176c9ec8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:21.186753 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11103299-09da-4fe3-a6dc-ef85176c9ec8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:21.187174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11103299-09da-4fe3-a6dc-ef85176c9ec8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:21.414086 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-11103299-09da-4fe3-a6dc-ef85176c9ec8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:21.421694 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-11103299-09da-4fe3-a6dc-ef85176c9ec8 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.248116 Oct 20 09:02:21.422588 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 153/282] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:21 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:21.929221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-02771deb-4866-413d-93e2-9aba0fc1f5ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:21.929837 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-02771deb-4866-413d-93e2-9aba0fc1f5ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:21.935827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-02771deb-4866-413d-93e2-9aba0fc1f5ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:21.936301 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-02771deb-4866-413d-93e2-9aba0fc1f5ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:21.936727 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-02771deb-4866-413d-93e2-9aba0fc1f5ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:21.937396 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-02771deb-4866-413d-93e2-9aba0fc1f5ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:21.937842 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-02771deb-4866-413d-93e2-9aba0fc1f5ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:21.938255 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-02771deb-4866-413d-93e2-9aba0fc1f5ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:22.017187 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9c41c09c-86db-4914-9fb9-98281b739b27 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:22.017786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9c41c09c-86db-4914-9fb9-98281b739b27 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:22.024629 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c41c09c-86db-4914-9fb9-98281b739b27 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:22.025195 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c41c09c-86db-4914-9fb9-98281b739b27 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:22.025565 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c41c09c-86db-4914-9fb9-98281b739b27 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:22.026262 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c41c09c-86db-4914-9fb9-98281b739b27 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:22.026951 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c41c09c-86db-4914-9fb9-98281b739b27 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:22.027379 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9c41c09c-86db-4914-9fb9-98281b739b27 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:22.073346 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-02771deb-4866-413d-93e2-9aba0fc1f5ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:22.081589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-02771deb-4866-413d-93e2-9aba0fc1f5ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.156175 Oct 20 09:02:22.082728 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 130/283] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:21 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:22.165301 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9c41c09c-86db-4914-9fb9-98281b739b27 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:22.173789 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9c41c09c-86db-4914-9fb9-98281b739b27 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.160396 Oct 20 09:02:22.174376 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 154/284] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:22 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:22.444050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7bf7c9a0-7845-4588-b0e0-1e96b585b44f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:22.444656 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7bf7c9a0-7845-4588-b0e0-1e96b585b44f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:22.450736 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bf7c9a0-7845-4588-b0e0-1e96b585b44f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:22.451219 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bf7c9a0-7845-4588-b0e0-1e96b585b44f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:22.451651 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bf7c9a0-7845-4588-b0e0-1e96b585b44f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:22.452340 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bf7c9a0-7845-4588-b0e0-1e96b585b44f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:22.452793 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bf7c9a0-7845-4588-b0e0-1e96b585b44f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:22.453214 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bf7c9a0-7845-4588-b0e0-1e96b585b44f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:22.584157 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7bf7c9a0-7845-4588-b0e0-1e96b585b44f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:22.592002 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7bf7c9a0-7845-4588-b0e0-1e96b585b44f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.151915 Oct 20 09:02:22.592575 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 131/285] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:22 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:23.105742 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-10b8a6e9-32a0-4709-9837-fa76cc511047 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:23.106350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-10b8a6e9-32a0-4709-9837-fa76cc511047 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:23.112818 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-10b8a6e9-32a0-4709-9837-fa76cc511047 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:23.113303 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-10b8a6e9-32a0-4709-9837-fa76cc511047 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:23.114177 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-10b8a6e9-32a0-4709-9837-fa76cc511047 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:23.114406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-10b8a6e9-32a0-4709-9837-fa76cc511047 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:23.114879 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-10b8a6e9-32a0-4709-9837-fa76cc511047 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:23.115315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-10b8a6e9-32a0-4709-9837-fa76cc511047 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:23.197103 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-43dcdec9-0e3e-4e8e-bc1f-2f93c2632797 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:23.197723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-43dcdec9-0e3e-4e8e-bc1f-2f93c2632797 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:23.204088 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43dcdec9-0e3e-4e8e-bc1f-2f93c2632797 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:23.204581 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43dcdec9-0e3e-4e8e-bc1f-2f93c2632797 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:23.205007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43dcdec9-0e3e-4e8e-bc1f-2f93c2632797 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:23.205693 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43dcdec9-0e3e-4e8e-bc1f-2f93c2632797 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:23.206141 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43dcdec9-0e3e-4e8e-bc1f-2f93c2632797 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:23.206582 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43dcdec9-0e3e-4e8e-bc1f-2f93c2632797 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:23.348965 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-43dcdec9-0e3e-4e8e-bc1f-2f93c2632797 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:23.357519 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-43dcdec9-0e3e-4e8e-bc1f-2f93c2632797 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.164596 Oct 20 09:02:23.358113 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 132/286] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:23 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:23.526073 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-10b8a6e9-32a0-4709-9837-fa76cc511047 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:23.534193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-10b8a6e9-32a0-4709-9837-fa76cc511047 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.432197 Oct 20 09:02:23.535115 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 155/287] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:23 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:23.615480 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-37412f81-525d-41ee-8d76-c097b4def5e4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:23.615986 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-37412f81-525d-41ee-8d76-c097b4def5e4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:23.622367 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37412f81-525d-41ee-8d76-c097b4def5e4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:23.622872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37412f81-525d-41ee-8d76-c097b4def5e4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:23.623303 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37412f81-525d-41ee-8d76-c097b4def5e4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:23.623971 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37412f81-525d-41ee-8d76-c097b4def5e4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:23.624410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37412f81-525d-41ee-8d76-c097b4def5e4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:23.624822 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37412f81-525d-41ee-8d76-c097b4def5e4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:23.754098 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-37412f81-525d-41ee-8d76-c097b4def5e4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:23.762274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-37412f81-525d-41ee-8d76-c097b4def5e4 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.150661 Oct 20 09:02:23.762871 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 133/288] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:23 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:24.383172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3b77c354-2252-4ece-bdd7-c7f8fcd0ff93 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:24.383926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3b77c354-2252-4ece-bdd7-c7f8fcd0ff93 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:24.390151 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b77c354-2252-4ece-bdd7-c7f8fcd0ff93 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:24.390672 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b77c354-2252-4ece-bdd7-c7f8fcd0ff93 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:24.391100 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b77c354-2252-4ece-bdd7-c7f8fcd0ff93 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:24.391788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b77c354-2252-4ece-bdd7-c7f8fcd0ff93 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:24.392231 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b77c354-2252-4ece-bdd7-c7f8fcd0ff93 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:24.392645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b77c354-2252-4ece-bdd7-c7f8fcd0ff93 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:24.519976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3b77c354-2252-4ece-bdd7-c7f8fcd0ff93 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:24.528092 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3b77c354-2252-4ece-bdd7-c7f8fcd0ff93 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.151474 Oct 20 09:02:24.528661 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 156/289] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:24 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:24.556778 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-98250dbc-007a-4e5e-b728-b8c1de75d04a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:24.557380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-98250dbc-007a-4e5e-b728-b8c1de75d04a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:24.563488 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98250dbc-007a-4e5e-b728-b8c1de75d04a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:24.563961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98250dbc-007a-4e5e-b728-b8c1de75d04a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:24.564393 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98250dbc-007a-4e5e-b728-b8c1de75d04a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:24.565074 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98250dbc-007a-4e5e-b728-b8c1de75d04a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:24.565517 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98250dbc-007a-4e5e-b728-b8c1de75d04a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:24.565942 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98250dbc-007a-4e5e-b728-b8c1de75d04a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:24.698389 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-98250dbc-007a-4e5e-b728-b8c1de75d04a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:24.700814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:24.707813 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-98250dbc-007a-4e5e-b728-b8c1de75d04a tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.154810 Oct 20 09:02:24.708587 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 134/290] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:24 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:24.785254 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5c7d966c-26b8-4def-a1af-8f79439ad58c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:24.785870 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5c7d966c-26b8-4def-a1af-8f79439ad58c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:24.792411 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c7d966c-26b8-4def-a1af-8f79439ad58c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:24.792917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c7d966c-26b8-4def-a1af-8f79439ad58c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:24.793341 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c7d966c-26b8-4def-a1af-8f79439ad58c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:24.794033 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c7d966c-26b8-4def-a1af-8f79439ad58c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:24.794524 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c7d966c-26b8-4def-a1af-8f79439ad58c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:24.794949 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c7d966c-26b8-4def-a1af-8f79439ad58c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:24.925657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5c7d966c-26b8-4def-a1af-8f79439ad58c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:24.933878 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5c7d966c-26b8-4def-a1af-8f79439ad58c tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.152888 Oct 20 09:02:24.934440 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 157/291] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:24 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:25.555687 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a279be49-93ab-46fe-aeb9-310c7d2f4f5d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:25.556287 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a279be49-93ab-46fe-aeb9-310c7d2f4f5d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:25.564360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a279be49-93ab-46fe-aeb9-310c7d2f4f5d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:25.565432 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a279be49-93ab-46fe-aeb9-310c7d2f4f5d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:25.566582 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a279be49-93ab-46fe-aeb9-310c7d2f4f5d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:25.567290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a279be49-93ab-46fe-aeb9-310c7d2f4f5d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:25.568306 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a279be49-93ab-46fe-aeb9-310c7d2f4f5d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:25.568758 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a279be49-93ab-46fe-aeb9-310c7d2f4f5d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:25.696665 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a279be49-93ab-46fe-aeb9-310c7d2f4f5d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:25.704179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a279be49-93ab-46fe-aeb9-310c7d2f4f5d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.152956 Oct 20 09:02:25.704733 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 135/292] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:25 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:25.732021 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0abfbb2a-4123-497d-be92-bb13e8655bab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:25.732626 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0abfbb2a-4123-497d-be92-bb13e8655bab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:25.739290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0abfbb2a-4123-497d-be92-bb13e8655bab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:25.739769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0abfbb2a-4123-497d-be92-bb13e8655bab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:25.740199 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0abfbb2a-4123-497d-be92-bb13e8655bab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:25.740888 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0abfbb2a-4123-497d-be92-bb13e8655bab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:25.741372 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0abfbb2a-4123-497d-be92-bb13e8655bab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:25.741788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0abfbb2a-4123-497d-be92-bb13e8655bab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:25.867983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0abfbb2a-4123-497d-be92-bb13e8655bab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:25.875501 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0abfbb2a-4123-497d-be92-bb13e8655bab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.147486 Oct 20 09:02:25.876064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 158/293] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:25 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:25.957222 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-da6eba6b-2caf-4f4f-a19c-95fb68576cae tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:25.958211 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-da6eba6b-2caf-4f4f-a19c-95fb68576cae tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:25.967862 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da6eba6b-2caf-4f4f-a19c-95fb68576cae tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:25.968376 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da6eba6b-2caf-4f4f-a19c-95fb68576cae tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:25.968803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da6eba6b-2caf-4f4f-a19c-95fb68576cae tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:25.969495 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da6eba6b-2caf-4f4f-a19c-95fb68576cae tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:25.969940 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da6eba6b-2caf-4f4f-a19c-95fb68576cae tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:25.970353 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da6eba6b-2caf-4f4f-a19c-95fb68576cae tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:26.104197 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-da6eba6b-2caf-4f4f-a19c-95fb68576cae tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:26.112278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-da6eba6b-2caf-4f4f-a19c-95fb68576cae tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.159723 Oct 20 09:02:26.112855 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 136/294] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:25 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:26.465731 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-540e30ae-6e89-400c-8617-510c54e09a5d req-dd723bf3-9786-4418-9896-397a841a2ad7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "df3974e9-9d16-4e6b-ac63-0f1257ff6d7c", "name": "network-vif-plugged", "status": "completed", "tag": "733a3a52-e016-43f3-9379-3f4b29beac85"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:26.476181 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-540e30ae-6e89-400c-8617-510c54e09a5d req-dd723bf3-9786-4418-9896-397a841a2ad7 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:26.476588 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-540e30ae-6e89-400c-8617-510c54e09a5d req-dd723bf3-9786-4418-9896-397a841a2ad7 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:26.477034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-540e30ae-6e89-400c-8617-510c54e09a5d req-dd723bf3-9786-4418-9896-397a841a2ad7 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:26.488011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-540e30ae-6e89-400c-8617-510c54e09a5d req-dd723bf3-9786-4418-9896-397a841a2ad7 service nova] Creating event network-vif-plugged:733a3a52-e016-43f3-9379-3f4b29beac85 for instance df3974e9-9d16-4e6b-ac63-0f1257ff6d7c on hv-cinder-maste Oct 20 09:02:26.492893 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-540e30ae-6e89-400c-8617-510c54e09a5d req-dd723bf3-9786-4418-9896-397a841a2ad7 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031335 Oct 20 09:02:26.493493 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 159/295] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:02:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:02:26.728372 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ad6d005f-4731-4668-a312-54e136ada420 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:26.728899 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ad6d005f-4731-4668-a312-54e136ada420 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:26.735273 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ad6d005f-4731-4668-a312-54e136ada420 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:26.735768 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ad6d005f-4731-4668-a312-54e136ada420 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:26.736206 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ad6d005f-4731-4668-a312-54e136ada420 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:26.736901 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ad6d005f-4731-4668-a312-54e136ada420 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:26.737357 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ad6d005f-4731-4668-a312-54e136ada420 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:26.737811 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ad6d005f-4731-4668-a312-54e136ada420 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:26.869665 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ad6d005f-4731-4668-a312-54e136ada420 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:26.877314 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ad6d005f-4731-4668-a312-54e136ada420 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.153520 Oct 20 09:02:26.877882 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 137/296] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:26 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:26.899737 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0a233dfa-dfdb-42de-b480-81f6f82cefef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:26.900421 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0a233dfa-dfdb-42de-b480-81f6f82cefef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:26.906686 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0a233dfa-dfdb-42de-b480-81f6f82cefef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:26.907251 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0a233dfa-dfdb-42de-b480-81f6f82cefef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:26.907893 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0a233dfa-dfdb-42de-b480-81f6f82cefef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:26.908626 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0a233dfa-dfdb-42de-b480-81f6f82cefef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:26.909306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0a233dfa-dfdb-42de-b480-81f6f82cefef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:26.909738 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0a233dfa-dfdb-42de-b480-81f6f82cefef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:27.039005 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0a233dfa-dfdb-42de-b480-81f6f82cefef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:27.046769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0a233dfa-dfdb-42de-b480-81f6f82cefef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.151822 Oct 20 09:02:27.047319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 160/297] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:26 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:27.138071 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-77eaf146-e595-4475-9014-28b90efa2cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:27.138715 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-77eaf146-e595-4475-9014-28b90efa2cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:27.144575 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77eaf146-e595-4475-9014-28b90efa2cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:27.145054 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77eaf146-e595-4475-9014-28b90efa2cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:27.145487 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77eaf146-e595-4475-9014-28b90efa2cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:27.146397 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77eaf146-e595-4475-9014-28b90efa2cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:27.146925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77eaf146-e595-4475-9014-28b90efa2cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:27.148072 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77eaf146-e595-4475-9014-28b90efa2cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:27.275119 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-77eaf146-e595-4475-9014-28b90efa2cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:27.282535 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-77eaf146-e595-4475-9014-28b90efa2cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.149089 Oct 20 09:02:27.283095 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 138/298] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:27 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:27.900439 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c44d1efd-8dd6-445a-a081-c2e97e5ee11a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:27.901030 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c44d1efd-8dd6-445a-a081-c2e97e5ee11a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:27.907512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c44d1efd-8dd6-445a-a081-c2e97e5ee11a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:27.907991 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c44d1efd-8dd6-445a-a081-c2e97e5ee11a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:27.908422 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c44d1efd-8dd6-445a-a081-c2e97e5ee11a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:27.909098 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c44d1efd-8dd6-445a-a081-c2e97e5ee11a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:27.909545 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c44d1efd-8dd6-445a-a081-c2e97e5ee11a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:27.909964 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c44d1efd-8dd6-445a-a081-c2e97e5ee11a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:28.042185 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c44d1efd-8dd6-445a-a081-c2e97e5ee11a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:28.050041 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c44d1efd-8dd6-445a-a081-c2e97e5ee11a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.153512 Oct 20 09:02:28.050632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 161/299] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:27 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:28.069032 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-31e168d1-d660-41e3-9fff-0bf0e0af1bdd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:28.069660 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-31e168d1-d660-41e3-9fff-0bf0e0af1bdd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:28.075658 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31e168d1-d660-41e3-9fff-0bf0e0af1bdd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:28.076139 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31e168d1-d660-41e3-9fff-0bf0e0af1bdd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:28.076565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31e168d1-d660-41e3-9fff-0bf0e0af1bdd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:28.077238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31e168d1-d660-41e3-9fff-0bf0e0af1bdd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:28.077676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31e168d1-d660-41e3-9fff-0bf0e0af1bdd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:28.078089 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31e168d1-d660-41e3-9fff-0bf0e0af1bdd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:28.158813 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-eda2459e-7286-4926-9bf9-4f9d51513b49 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:28.159236 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-eda2459e-7286-4926-9bf9-4f9d51513b49 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:28.165329 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-eda2459e-7286-4926-9bf9-4f9d51513b49 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:28.165812 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-eda2459e-7286-4926-9bf9-4f9d51513b49 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:28.166247 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-eda2459e-7286-4926-9bf9-4f9d51513b49 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:28.166973 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-eda2459e-7286-4926-9bf9-4f9d51513b49 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:28.167424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-eda2459e-7286-4926-9bf9-4f9d51513b49 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:28.167849 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-eda2459e-7286-4926-9bf9-4f9d51513b49 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:28.191506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-eda2459e-7286-4926-9bf9-4f9d51513b49 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/ips" status: 200 len: 149 microversion: 2.1 time: 0.036780 Oct 20 09:02:28.199047 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 162/300] 10.222.0.36 () {58 vars in 1248 bytes} [Thu Oct 20 09:02:28 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/ips => generated 149 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:02:28.216036 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-31e168d1-d660-41e3-9fff-0bf0e0af1bdd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:28.224740 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-31e168d1-d660-41e3-9fff-0bf0e0af1bdd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.159551 Oct 20 09:02:28.225310 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 139/301] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:28 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:28.307971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-29aa2513-863b-453f-a804-965f2ce29cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:28.308939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-29aa2513-863b-453f-a804-965f2ce29cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:28.315062 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29aa2513-863b-453f-a804-965f2ce29cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:28.315732 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29aa2513-863b-453f-a804-965f2ce29cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:28.316168 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29aa2513-863b-453f-a804-965f2ce29cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:28.316862 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29aa2513-863b-453f-a804-965f2ce29cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:28.317316 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29aa2513-863b-453f-a804-965f2ce29cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:28.317764 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29aa2513-863b-453f-a804-965f2ce29cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:28.322601 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:28.478514 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-29aa2513-863b-453f-a804-965f2ce29cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:28.486144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-29aa2513-863b-453f-a804-965f2ce29cee tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1314 microversion: 2.1 time: 0.182506 Oct 20 09:02:28.486733 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 163/302] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:28 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1314 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:28.509083 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-5c5de552-65b8-4b18-83d3-918926429d20 req-9ffeb711-0e2a-491f-8e8f-7058da8ff9ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "26ae0069-be67-4921-aca3-4cfae5214cf5", "name": "network-vif-plugged", "status": "completed", "tag": "ead535c7-97fc-458e-b3a8-642e6ca66f0a"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:28.527043 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-5c5de552-65b8-4b18-83d3-918926429d20 req-9ffeb711-0e2a-491f-8e8f-7058da8ff9ac service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:28.527522 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-5c5de552-65b8-4b18-83d3-918926429d20 req-9ffeb711-0e2a-491f-8e8f-7058da8ff9ac service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:28.527943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-5c5de552-65b8-4b18-83d3-918926429d20 req-9ffeb711-0e2a-491f-8e8f-7058da8ff9ac service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:28.539790 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-5c5de552-65b8-4b18-83d3-918926429d20 req-9ffeb711-0e2a-491f-8e8f-7058da8ff9ac service nova] Creating event network-vif-plugged:ead535c7-97fc-458e-b3a8-642e6ca66f0a for instance 26ae0069-be67-4921-aca3-4cfae5214cf5 on hv-cinder-maste Oct 20 09:02:28.545075 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-5c5de552-65b8-4b18-83d3-918926429d20 req-9ffeb711-0e2a-491f-8e8f-7058da8ff9ac service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041096 Oct 20 09:02:28.545780 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 140/303] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:02:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:02:29.074110 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7a83630b-2ddb-4b75-ac9a-8452315d3c18 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:29.074690 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7a83630b-2ddb-4b75-ac9a-8452315d3c18 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:29.081016 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a83630b-2ddb-4b75-ac9a-8452315d3c18 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:29.081488 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a83630b-2ddb-4b75-ac9a-8452315d3c18 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:29.081914 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a83630b-2ddb-4b75-ac9a-8452315d3c18 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:29.082617 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a83630b-2ddb-4b75-ac9a-8452315d3c18 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:29.083060 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a83630b-2ddb-4b75-ac9a-8452315d3c18 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:29.083475 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7a83630b-2ddb-4b75-ac9a-8452315d3c18 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:29.222735 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7a83630b-2ddb-4b75-ac9a-8452315d3c18 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:29.230532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7a83630b-2ddb-4b75-ac9a-8452315d3c18 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.161200 Oct 20 09:02:29.231320 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 164/304] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:29 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:29.248162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9b36092c-3a9b-4cd1-904a-eecb066a8878 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:29.248901 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-9b36092c-3a9b-4cd1-904a-eecb066a8878 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:29.255068 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9b36092c-3a9b-4cd1-904a-eecb066a8878 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:29.255601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9b36092c-3a9b-4cd1-904a-eecb066a8878 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:29.256034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9b36092c-3a9b-4cd1-904a-eecb066a8878 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:29.257224 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9b36092c-3a9b-4cd1-904a-eecb066a8878 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:29.257676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9b36092c-3a9b-4cd1-904a-eecb066a8878 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:29.258145 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9b36092c-3a9b-4cd1-904a-eecb066a8878 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:29.391140 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-9b36092c-3a9b-4cd1-904a-eecb066a8878 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:29.399397 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-9b36092c-3a9b-4cd1-904a-eecb066a8878 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.155555 Oct 20 09:02:29.399961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 141/305] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:29 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:29.509580 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3e29a2f9-f121-4ba1-b464-c2a29f56f06b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:29.510189 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3e29a2f9-f121-4ba1-b464-c2a29f56f06b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:29.516232 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3e29a2f9-f121-4ba1-b464-c2a29f56f06b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:29.516713 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3e29a2f9-f121-4ba1-b464-c2a29f56f06b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:29.517162 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3e29a2f9-f121-4ba1-b464-c2a29f56f06b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:29.517822 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3e29a2f9-f121-4ba1-b464-c2a29f56f06b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:29.518273 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3e29a2f9-f121-4ba1-b464-c2a29f56f06b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:29.518721 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3e29a2f9-f121-4ba1-b464-c2a29f56f06b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:29.630407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9a31b8c7-e484-4044-b279-eeee293db87c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "73bd08b5-9c2e-40b7-92db-80f1886711b9"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:29.632826 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-9a31b8c7-e484-4044-b279-eeee293db87c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:29.638709 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9a31b8c7-e484-4044-b279-eeee293db87c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:29.639194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9a31b8c7-e484-4044-b279-eeee293db87c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:29.639635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9a31b8c7-e484-4044-b279-eeee293db87c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:29.640322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9a31b8c7-e484-4044-b279-eeee293db87c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:29.640774 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9a31b8c7-e484-4044-b279-eeee293db87c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:29.641196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9a31b8c7-e484-4044-b279-eeee293db87c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:29.645657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3e29a2f9-f121-4ba1-b464-c2a29f56f06b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:29.654099 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3e29a2f9-f121-4ba1-b464-c2a29f56f06b tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1498 microversion: 2.1 time: 0.148882 Oct 20 09:02:29.663636 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 165/306] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:29 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1498 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:30.192926 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-9a31b8c7-e484-4044-b279-eeee293db87c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.566239 Oct 20 09:02:30.193544 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 142/307] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:02:29 2022] POST /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments => generated 194 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:02:30.254343 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fbcafc88-0ba5-49d9-a7ad-e14b8f94a5e1 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:30.254983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fbcafc88-0ba5-49d9-a7ad-e14b8f94a5e1 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:30.261490 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbcafc88-0ba5-49d9-a7ad-e14b8f94a5e1 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:30.261964 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbcafc88-0ba5-49d9-a7ad-e14b8f94a5e1 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:30.262390 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbcafc88-0ba5-49d9-a7ad-e14b8f94a5e1 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:30.263100 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbcafc88-0ba5-49d9-a7ad-e14b8f94a5e1 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:30.263547 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbcafc88-0ba5-49d9-a7ad-e14b8f94a5e1 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:30.263964 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbcafc88-0ba5-49d9-a7ad-e14b8f94a5e1 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:30.400546 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-fbcafc88-0ba5-49d9-a7ad-e14b8f94a5e1 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:30.408113 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-fbcafc88-0ba5-49d9-a7ad-e14b8f94a5e1 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.157647 Oct 20 09:02:30.408670 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 166/308] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:30 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:30.423147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fc0ed33f-6ae8-47cd-8dcf-107f38783a09 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:30.423836 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-fc0ed33f-6ae8-47cd-8dcf-107f38783a09 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:30.430654 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc0ed33f-6ae8-47cd-8dcf-107f38783a09 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:30.431146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc0ed33f-6ae8-47cd-8dcf-107f38783a09 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:30.431582 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc0ed33f-6ae8-47cd-8dcf-107f38783a09 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:30.432285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc0ed33f-6ae8-47cd-8dcf-107f38783a09 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:30.432733 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc0ed33f-6ae8-47cd-8dcf-107f38783a09 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:30.433161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc0ed33f-6ae8-47cd-8dcf-107f38783a09 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:30.454247 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6bf71f45-f28e-43e9-a35e-8ec5abb9eff1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:30.454821 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6bf71f45-f28e-43e9-a35e-8ec5abb9eff1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:30.461777 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bf71f45-f28e-43e9-a35e-8ec5abb9eff1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:30.462270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bf71f45-f28e-43e9-a35e-8ec5abb9eff1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:30.462726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bf71f45-f28e-43e9-a35e-8ec5abb9eff1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:30.463433 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bf71f45-f28e-43e9-a35e-8ec5abb9eff1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:30.463876 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bf71f45-f28e-43e9-a35e-8ec5abb9eff1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:30.464306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6bf71f45-f28e-43e9-a35e-8ec5abb9eff1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:30.492608 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6bf71f45-f28e-43e9-a35e-8ec5abb9eff1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:02:30.583042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6bf71f45-f28e-43e9-a35e-8ec5abb9eff1 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "DELETE /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 204 len: 0 microversion: 2.1 time: 0.132348 Oct 20 09:02:30.586742 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 167/309] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:02:30 2022] DELETE /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 20 09:02:30.593024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e9ccff54-1520-4d12-91fb-8426bc71cd96 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:30.593732 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-e9ccff54-1520-4d12-91fb-8426bc71cd96 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:30.599594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e9ccff54-1520-4d12-91fb-8426bc71cd96 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:30.600073 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e9ccff54-1520-4d12-91fb-8426bc71cd96 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:30.601098 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e9ccff54-1520-4d12-91fb-8426bc71cd96 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:30.601782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e9ccff54-1520-4d12-91fb-8426bc71cd96 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:30.602220 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e9ccff54-1520-4d12-91fb-8426bc71cd96 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:30.602691 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e9ccff54-1520-4d12-91fb-8426bc71cd96 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:30.726893 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-fc0ed33f-6ae8-47cd-8dcf-107f38783a09 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:30.734639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fc0ed33f-6ae8-47cd-8dcf-107f38783a09 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.315533 Oct 20 09:02:30.735220 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 143/310] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:30 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:30.737765 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e9ccff54-1520-4d12-91fb-8426bc71cd96 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:30.746828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e9ccff54-1520-4d12-91fb-8426bc71cd96 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1504 microversion: 2.1 time: 0.158178 Oct 20 09:02:30.747379 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 168/311] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:30 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1504 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:31.432116 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d55b0fb9-59ec-46d0-88ec-5b98f4521ba7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:31.432842 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d55b0fb9-59ec-46d0-88ec-5b98f4521ba7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:31.439661 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55b0fb9-59ec-46d0-88ec-5b98f4521ba7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:31.440173 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55b0fb9-59ec-46d0-88ec-5b98f4521ba7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:31.440610 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55b0fb9-59ec-46d0-88ec-5b98f4521ba7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:31.441304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55b0fb9-59ec-46d0-88ec-5b98f4521ba7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:31.441756 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55b0fb9-59ec-46d0-88ec-5b98f4521ba7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:31.442181 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d55b0fb9-59ec-46d0-88ec-5b98f4521ba7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:31.570574 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d55b0fb9-59ec-46d0-88ec-5b98f4521ba7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:31.578194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d55b0fb9-59ec-46d0-88ec-5b98f4521ba7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.149975 Oct 20 09:02:31.578777 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 144/312] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:31 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:31.758397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2fa7d046-8779-4d98-8641-e61cdb7820f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:31.759017 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2fa7d046-8779-4d98-8641-e61cdb7820f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:31.765335 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2fa7d046-8779-4d98-8641-e61cdb7820f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:31.765806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2fa7d046-8779-4d98-8641-e61cdb7820f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:31.766231 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2fa7d046-8779-4d98-8641-e61cdb7820f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:31.766950 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2fa7d046-8779-4d98-8641-e61cdb7820f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:31.767393 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2fa7d046-8779-4d98-8641-e61cdb7820f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:31.767990 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2fa7d046-8779-4d98-8641-e61cdb7820f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:31.772692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0421adef-ec20-4d49-8ac6-9895fa8dd681 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:31.773294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0421adef-ec20-4d49-8ac6-9895fa8dd681 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:31.779085 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0421adef-ec20-4d49-8ac6-9895fa8dd681 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:31.779563 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0421adef-ec20-4d49-8ac6-9895fa8dd681 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:31.779999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0421adef-ec20-4d49-8ac6-9895fa8dd681 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:31.780682 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0421adef-ec20-4d49-8ac6-9895fa8dd681 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:31.781131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0421adef-ec20-4d49-8ac6-9895fa8dd681 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:31.781552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0421adef-ec20-4d49-8ac6-9895fa8dd681 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:31.899727 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2fa7d046-8779-4d98-8641-e61cdb7820f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:31.907329 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2fa7d046-8779-4d98-8641-e61cdb7820f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.153340 Oct 20 09:02:31.907899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 169/313] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:31 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:31.916977 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0421adef-ec20-4d49-8ac6-9895fa8dd681 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:31.924820 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0421adef-ec20-4d49-8ac6-9895fa8dd681 tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1504 microversion: 2.1 time: 0.155890 Oct 20 09:02:31.925758 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 145/314] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:31 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1504 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:32.220030 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-337c1d9c-90e5-4590-ba31-63eb55497fd6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:32.220518 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-337c1d9c-90e5-4590-ba31-63eb55497fd6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:32.226492 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-337c1d9c-90e5-4590-ba31-63eb55497fd6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:32.226975 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-337c1d9c-90e5-4590-ba31-63eb55497fd6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:32.227407 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-337c1d9c-90e5-4590-ba31-63eb55497fd6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:32.228089 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-337c1d9c-90e5-4590-ba31-63eb55497fd6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:32.228538 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-337c1d9c-90e5-4590-ba31-63eb55497fd6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:32.228960 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-337c1d9c-90e5-4590-ba31-63eb55497fd6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:32.251228 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-337c1d9c-90e5-4590-ba31-63eb55497fd6 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/ips" status: 200 len: 150 microversion: 2.20 time: 0.035096 Oct 20 09:02:32.251777 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 170/315] 10.222.0.36 () {60 vars in 1289 bytes} [Thu Oct 20 09:02:32 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/ips => generated 150 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 09:02:32.602865 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a8de28a4-4caa-4930-b84e-9d0ec795a491 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:32.603491 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a8de28a4-4caa-4930-b84e-9d0ec795a491 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:32.609726 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a8de28a4-4caa-4930-b84e-9d0ec795a491 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:32.610101 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a8de28a4-4caa-4930-b84e-9d0ec795a491 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:32.610554 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a8de28a4-4caa-4930-b84e-9d0ec795a491 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:32.611232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a8de28a4-4caa-4930-b84e-9d0ec795a491 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:32.611674 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a8de28a4-4caa-4930-b84e-9d0ec795a491 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:32.612088 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a8de28a4-4caa-4930-b84e-9d0ec795a491 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:32.747686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a8de28a4-4caa-4930-b84e-9d0ec795a491 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:32.755272 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a8de28a4-4caa-4930-b84e-9d0ec795a491 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1362 microversion: 2.1 time: 0.156265 Oct 20 09:02:32.755834 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 146/316] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:32 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1362 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:32.931366 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c68af4ac-558b-4c68-a7c1-f12121fcf64f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:32.931911 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c68af4ac-558b-4c68-a7c1-f12121fcf64f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:32.938264 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c68af4ac-558b-4c68-a7c1-f12121fcf64f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:32.938787 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c68af4ac-558b-4c68-a7c1-f12121fcf64f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:32.939228 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c68af4ac-558b-4c68-a7c1-f12121fcf64f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:32.939919 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c68af4ac-558b-4c68-a7c1-f12121fcf64f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:32.940435 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c68af4ac-558b-4c68-a7c1-f12121fcf64f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:32.940866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c68af4ac-558b-4c68-a7c1-f12121fcf64f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:32.949157 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2dc49dca-bcd8-482e-be0a-70b0fe7b7a2e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:32.949773 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2dc49dca-bcd8-482e-be0a-70b0fe7b7a2e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:32.956016 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2dc49dca-bcd8-482e-be0a-70b0fe7b7a2e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:32.956504 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2dc49dca-bcd8-482e-be0a-70b0fe7b7a2e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:32.956937 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2dc49dca-bcd8-482e-be0a-70b0fe7b7a2e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:32.957622 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2dc49dca-bcd8-482e-be0a-70b0fe7b7a2e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:32.958077 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2dc49dca-bcd8-482e-be0a-70b0fe7b7a2e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:32.958531 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2dc49dca-bcd8-482e-be0a-70b0fe7b7a2e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:33.080991 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c68af4ac-558b-4c68-a7c1-f12121fcf64f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:33.089135 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c68af4ac-558b-4c68-a7c1-f12121fcf64f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.162392 Oct 20 09:02:33.090036 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 171/317] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:32 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:33.097619 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2dc49dca-bcd8-482e-be0a-70b0fe7b7a2e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:33.106851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2dc49dca-bcd8-482e-be0a-70b0fe7b7a2e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1504 microversion: 2.1 time: 0.161645 Oct 20 09:02:33.107773 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 147/318] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:32 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1504 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:33.778730 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6c2e253e-2e98-4eef-8026-deee152b8fa0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:33.779339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6c2e253e-2e98-4eef-8026-deee152b8fa0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:33.786537 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c2e253e-2e98-4eef-8026-deee152b8fa0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:33.786537 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c2e253e-2e98-4eef-8026-deee152b8fa0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:33.786537 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c2e253e-2e98-4eef-8026-deee152b8fa0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:33.787053 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c2e253e-2e98-4eef-8026-deee152b8fa0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:33.787499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c2e253e-2e98-4eef-8026-deee152b8fa0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:33.787936 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c2e253e-2e98-4eef-8026-deee152b8fa0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:33.913236 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6c2e253e-2e98-4eef-8026-deee152b8fa0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:33.920860 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6c2e253e-2e98-4eef-8026-deee152b8fa0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1548 microversion: 2.1 time: 0.146000 Oct 20 09:02:33.921414 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 172/319] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:33 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1548 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:33.943204 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a3a75dc1-0be9-4b83-8ed2-c51c708c6b60 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:33.943698 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a3a75dc1-0be9-4b83-8ed2-c51c708c6b60 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:33.946686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:33.951882 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3a75dc1-0be9-4b83-8ed2-c51c708c6b60 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:33.952363 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3a75dc1-0be9-4b83-8ed2-c51c708c6b60 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:33.952799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3a75dc1-0be9-4b83-8ed2-c51c708c6b60 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:33.953494 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3a75dc1-0be9-4b83-8ed2-c51c708c6b60 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:33.953944 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3a75dc1-0be9-4b83-8ed2-c51c708c6b60 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:33.954367 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3a75dc1-0be9-4b83-8ed2-c51c708c6b60 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:34.033792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a3a75dc1-0be9-4b83-8ed2-c51c708c6b60 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.094481 Oct 20 09:02:34.034382 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 148/320] 10.222.0.36 () {58 vars in 1275 bytes} [Thu Oct 20 09:02:33 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:02:34.112306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c41a4e88-bb17-4002-a4e3-7c4280cbfc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:34.112885 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c41a4e88-bb17-4002-a4e3-7c4280cbfc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:34.120284 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c41a4e88-bb17-4002-a4e3-7c4280cbfc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:34.121023 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c41a4e88-bb17-4002-a4e3-7c4280cbfc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:34.121793 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c41a4e88-bb17-4002-a4e3-7c4280cbfc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:34.122867 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c41a4e88-bb17-4002-a4e3-7c4280cbfc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:34.123475 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c41a4e88-bb17-4002-a4e3-7c4280cbfc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:34.124059 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c41a4e88-bb17-4002-a4e3-7c4280cbfc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:34.133160 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d0aa70b4-4ea8-452f-a23b-d949ba00ab8e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:34.133960 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d0aa70b4-4ea8-452f-a23b-d949ba00ab8e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:34.141585 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d0aa70b4-4ea8-452f-a23b-d949ba00ab8e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:34.142310 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d0aa70b4-4ea8-452f-a23b-d949ba00ab8e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:34.142966 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d0aa70b4-4ea8-452f-a23b-d949ba00ab8e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:34.143824 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d0aa70b4-4ea8-452f-a23b-d949ba00ab8e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:34.144423 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d0aa70b4-4ea8-452f-a23b-d949ba00ab8e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:34.145189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d0aa70b4-4ea8-452f-a23b-d949ba00ab8e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:34.397284 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d0aa70b4-4ea8-452f-a23b-d949ba00ab8e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:34.405539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d0aa70b4-4ea8-452f-a23b-d949ba00ab8e tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1504 microversion: 2.1 time: 0.278857 Oct 20 09:02:34.406310 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 149/321] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:34 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1504 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:34.421286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c41a4e88-bb17-4002-a4e3-7c4280cbfc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:34.434920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c41a4e88-bb17-4002-a4e3-7c4280cbfc49 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.325377 Oct 20 09:02:34.434920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 173/322] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:34 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:34.754432 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-6bb444d0-8e3a-4642-9ac0-a55b807531c9 req-ed0ecf30-8a87-42ec-bb98-cfb9f3d9f238 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "df3974e9-9d16-4e6b-ac63-0f1257ff6d7c", "name": "network-vif-deleted", "tag": "733a3a52-e016-43f3-9379-3f4b29beac85"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:34.764355 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6bb444d0-8e3a-4642-9ac0-a55b807531c9 req-ed0ecf30-8a87-42ec-bb98-cfb9f3d9f238 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:34.764863 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6bb444d0-8e3a-4642-9ac0-a55b807531c9 req-ed0ecf30-8a87-42ec-bb98-cfb9f3d9f238 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:34.765349 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6bb444d0-8e3a-4642-9ac0-a55b807531c9 req-ed0ecf30-8a87-42ec-bb98-cfb9f3d9f238 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:34.777739 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-6bb444d0-8e3a-4642-9ac0-a55b807531c9 req-ed0ecf30-8a87-42ec-bb98-cfb9f3d9f238 service nova] Creating event network-vif-deleted:733a3a52-e016-43f3-9379-3f4b29beac85 for instance df3974e9-9d16-4e6b-ac63-0f1257ff6d7c on hv-cinder-maste Oct 20 09:02:34.782504 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-6bb444d0-8e3a-4642-9ac0-a55b807531c9 req-ed0ecf30-8a87-42ec-bb98-cfb9f3d9f238 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032162 Oct 20 09:02:34.783650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 150/323] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:02:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:02:35.125224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-6e62bbe1-fb17-4f02-8865-1564f68050b8 req-4efece78-6925-43a9-8d71-f394f5bb5544 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "26ae0069-be67-4921-aca3-4cfae5214cf5", "tag": "ead535c7-97fc-458e-b3a8-642e6ca66f0a"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:35.137193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-6e62bbe1-fb17-4f02-8865-1564f68050b8 req-4efece78-6925-43a9-8d71-f394f5bb5544 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:35.137786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-6e62bbe1-fb17-4f02-8865-1564f68050b8 req-4efece78-6925-43a9-8d71-f394f5bb5544 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:35.138352 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-6e62bbe1-fb17-4f02-8865-1564f68050b8 req-4efece78-6925-43a9-8d71-f394f5bb5544 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:35.150776 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-6e62bbe1-fb17-4f02-8865-1564f68050b8 req-4efece78-6925-43a9-8d71-f394f5bb5544 service nova] Creating event network-changed:ead535c7-97fc-458e-b3a8-642e6ca66f0a for instance 26ae0069-be67-4921-aca3-4cfae5214cf5 on hv-cinder-maste Oct 20 09:02:35.155700 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-6e62bbe1-fb17-4f02-8865-1564f68050b8 req-4efece78-6925-43a9-8d71-f394f5bb5544 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034241 Oct 20 09:02:35.156490 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 174/324] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:02:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:02:35.206170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6748f1ca-7dc3-43f2-baf8-118d8765fc02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:35.207052 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6748f1ca-7dc3-43f2-baf8-118d8765fc02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:35.213855 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6748f1ca-7dc3-43f2-baf8-118d8765fc02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:35.214352 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6748f1ca-7dc3-43f2-baf8-118d8765fc02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:35.214826 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6748f1ca-7dc3-43f2-baf8-118d8765fc02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:35.215796 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6748f1ca-7dc3-43f2-baf8-118d8765fc02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:35.216252 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6748f1ca-7dc3-43f2-baf8-118d8765fc02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:35.216679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6748f1ca-7dc3-43f2-baf8-118d8765fc02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:35.239751 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-6748f1ca-7dc3-43f2-baf8-118d8765fc02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lazy-loading 'flavor' on Instance uuid f8d61907-24ca-4e74-917e-d218d0f89557 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:02:35.243302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:35.433484 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:35.434379 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:35.442553 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:35.443323 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:35.443757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:35.445615 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:35.446225 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:35.446825 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:35.466334 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6748f1ca-7dc3-43f2-baf8-118d8765fc02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "POST /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/action" status: 202 len: 0 microversion: 2.20 time: 0.264772 Oct 20 09:02:35.467303 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 151/325] 10.222.0.36 () {62 vars in 1319 bytes} [Thu Oct 20 09:02:35 2022] POST /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/action => generated 0 bytes in 267 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:02:35.473520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a05722dd-670c-4837-b1a7-cc65de641cab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:35.474220 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a05722dd-670c-4837-b1a7-cc65de641cab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:35.481333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a05722dd-670c-4837-b1a7-cc65de641cab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:35.481975 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a05722dd-670c-4837-b1a7-cc65de641cab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:35.482639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a05722dd-670c-4837-b1a7-cc65de641cab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:35.483766 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a05722dd-670c-4837-b1a7-cc65de641cab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:35.485108 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a05722dd-670c-4837-b1a7-cc65de641cab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:35.485703 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a05722dd-670c-4837-b1a7-cc65de641cab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:35.497223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:35.497632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:35.498240 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:35.509843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lazy-loading 'fault' on Instance uuid df3974e9-9d16-4e6b-ac63-0f1257ff6d7c {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:02:35.634483 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '930afcdb977f4ed2a33b366563a755f1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '847f0b8423334fc7b505313a10845bcb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:35.643537 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e66b248f-2cfc-4ef1-9077-30bf396ffcaf tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 200 len: 1304 microversion: 2.1 time: 0.216619 Oct 20 09:02:35.648560 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a05722dd-670c-4837-b1a7-cc65de641cab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:35.657489 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a05722dd-670c-4837-b1a7-cc65de641cab tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.188849 Oct 20 09:02:35.658419 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 152/326] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:35 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:35.659150 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 175/327] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:35 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 1304 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:35.663070 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:35.663677 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:35.669738 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:35.670229 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:35.670697 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:35.671391 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:35.671949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:35.672377 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:35.799645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:35.808474 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:35.809918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:35.811134 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-36ba3b80-3543-47eb-aca5-06ab920c1247 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.151871 Oct 20 09:02:35.811696 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 153/328] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:35 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:36.667948 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:36.668550 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] [instance: df3974e9-9d16-4e6b-ac63-0f1257ff6d7c] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:36.675024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:36.675696 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:36.676127 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:36.676807 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:36.677251 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:36.677667 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:36.680535 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a5e22ab0-680c-4069-be7e-5be21b0f5f79 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:36.684674 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a5e22ab0-680c-4069-be7e-5be21b0f5f79 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:36.692929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] HTTP exception thrown: Instance df3974e9-9d16-4e6b-ac63-0f1257ff6d7c could not be found. Oct 20 09:02:36.694185 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5e22ab0-680c-4069-be7e-5be21b0f5f79 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Request method failure captured: Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c HTTP/1.1 Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ5qkfCJv3bhU1P64sw6kE5bR1OrRjcvm2iWVdCWnM7BxDdCmiN7RJHPdD8BF24tEsLwMHzLkQxLFGUGO2YfLi3Cw4793gFjP-nbgPyNrGnW5fsoGSmhc8sz8xwQuLNqn_B_Mio7_-f43SyWXa5UkAVfrSch1_VyQaNxl1WrrsboAQM Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 847f0b8423334fc7b505313a10845bcb Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-VolumesActionsTest-823875808 Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:02:36.694633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/847f0b8423334fc7b505313a10845bcb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_847f0b8423334fc7b505313a10845bcb", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/847f0b8423334fc7b505313a10845bcb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/847f0b8423334fc7b505313a10845bcb"}]}] Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-VolumesActionsTest-823875808 Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 847f0b8423334fc7b505313a10845bcb Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-VolumesActionsTest-823875808 Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-VolumesActionsTest-823875808-project Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 930afcdb977f4ed2a33b366563a755f1 Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-VolumesActionsTest-823875808-project Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance df3974e9-9d16-4e6b-ac63-0f1257ff6d7c could not be found. Oct 20 09:02:36.696147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': 'df3974e9-9d16-4e6b-ac63-0f1257ff6d7c'} Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance df3974e9-9d16-4e6b-ac63-0f1257ff6d7c could not be found. Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:02:36.696915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance df3974e9-9d16-4e6b-ac63-0f1257ff6d7c could not be found. Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance df3974e9-9d16-4e6b-ac63-0f1257ff6d7c could not be found. Oct 20 09:02:36.698144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:02:36.699291 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5e22ab0-680c-4069-be7e-5be21b0f5f79 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:36.699291 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5e22ab0-680c-4069-be7e-5be21b0f5f79 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:36.699452 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] Returning 404 to user: Instance df3974e9-9d16-4e6b-ac63-0f1257ff6d7c could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:02:36.699542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5e22ab0-680c-4069-be7e-5be21b0f5f79 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:36.699634 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-fbb086fa-f04d-41c5-9789-6c68ef0e3b7f tempest-VolumesActionsTest-823875808 tempest-VolumesActionsTest-823875808-project] 10.222.0.36 "GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c" status: 404 len: 111 microversion: 2.1 time: 0.031854 Oct 20 09:02:36.699724 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5e22ab0-680c-4069-be7e-5be21b0f5f79 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:36.699724 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5e22ab0-680c-4069-be7e-5be21b0f5f79 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:36.699890 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 176/329] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:36 2022] GET /compute/v2.1/servers/df3974e9-9d16-4e6b-ac63-0f1257ff6d7c => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:02:36.833995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a5e22ab0-680c-4069-be7e-5be21b0f5f79 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:36.837196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:36.838387 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:36.842626 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a5e22ab0-680c-4069-be7e-5be21b0f5f79 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.166277 Oct 20 09:02:36.843559 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 154/330] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:36 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:36.848295 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:36.848862 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:36.849368 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:36.849985 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:36.850987 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:36.851407 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:37.003103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:37.011888 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:37.013206 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:37.014354 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7bef19c6-3eb3-4480-a32c-a41e70807858 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.181248 Oct 20 09:02:37.014930 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 177/331] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:36 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:02:37.865845 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bd49f2a2-6fea-46c3-b8fd-95aa2a711226 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:37.866585 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bd49f2a2-6fea-46c3-b8fd-95aa2a711226 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:37.872910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd49f2a2-6fea-46c3-b8fd-95aa2a711226 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:37.873380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd49f2a2-6fea-46c3-b8fd-95aa2a711226 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:37.873806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd49f2a2-6fea-46c3-b8fd-95aa2a711226 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:37.874502 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd49f2a2-6fea-46c3-b8fd-95aa2a711226 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:37.874945 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd49f2a2-6fea-46c3-b8fd-95aa2a711226 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:37.875363 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd49f2a2-6fea-46c3-b8fd-95aa2a711226 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:38.000991 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bd49f2a2-6fea-46c3-b8fd-95aa2a711226 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:38.008593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bd49f2a2-6fea-46c3-b8fd-95aa2a711226 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.146663 Oct 20 09:02:38.009142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 155/332] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:37 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:38.040073 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:38.040664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:38.046947 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:38.047414 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:38.047836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:38.048505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:38.048947 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:38.049362 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:38.175315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:38.183351 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:38.184805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:38.185955 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2d40563e-70ee-4afd-b680-da3e638093f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.150259 Oct 20 09:02:38.186526 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 178/333] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:38 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:39.030424 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f877fe1b-16cd-4c70-8c90-357912227bc2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:39.031129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f877fe1b-16cd-4c70-8c90-357912227bc2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:39.037405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f877fe1b-16cd-4c70-8c90-357912227bc2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:39.037781 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f877fe1b-16cd-4c70-8c90-357912227bc2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:39.038212 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f877fe1b-16cd-4c70-8c90-357912227bc2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:39.039562 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f877fe1b-16cd-4c70-8c90-357912227bc2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:39.040027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f877fe1b-16cd-4c70-8c90-357912227bc2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:39.040445 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f877fe1b-16cd-4c70-8c90-357912227bc2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:39.172573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f877fe1b-16cd-4c70-8c90-357912227bc2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:39.181161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f877fe1b-16cd-4c70-8c90-357912227bc2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.154475 Oct 20 09:02:39.181847 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 156/334] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:39 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:39.212355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:39.212988 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:39.219619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:39.220098 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:39.220528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:39.221214 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:39.221835 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:39.222256 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:39.240313 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:39.351918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:39.359967 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:39.361410 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:39.362588 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-625a7ec4-260a-44e4-96dc-f9d0decf362e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.154615 Oct 20 09:02:39.363141 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 179/335] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:39 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:40.205298 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-074a1c0d-19f4-4636-bf03-21fa55b317fd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:40.205897 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-074a1c0d-19f4-4636-bf03-21fa55b317fd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:40.214290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-074a1c0d-19f4-4636-bf03-21fa55b317fd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:40.214806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-074a1c0d-19f4-4636-bf03-21fa55b317fd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:40.215539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-074a1c0d-19f4-4636-bf03-21fa55b317fd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:40.216281 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-074a1c0d-19f4-4636-bf03-21fa55b317fd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:40.216730 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-074a1c0d-19f4-4636-bf03-21fa55b317fd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:40.217139 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-074a1c0d-19f4-4636-bf03-21fa55b317fd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:40.352049 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-074a1c0d-19f4-4636-bf03-21fa55b317fd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:40.359655 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-074a1c0d-19f4-4636-bf03-21fa55b317fd tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.158369 Oct 20 09:02:40.360251 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 157/336] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:40 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:40.388124 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:40.389241 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:40.396067 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:40.396550 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:40.396981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:40.397667 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:40.398939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:40.399367 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:40.530351 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:40.539076 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:40.540530 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:40.541806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2e6d6973-2085-4b30-b0fe-aa6a76f4709a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.157629 Oct 20 09:02:40.542354 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 180/337] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:40 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:41.178007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-64b1f39e-23de-4738-9967-5d7e4949cee0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f320b8eb-5558-4511-bf09-b963d2f19942", "device": "/dev/sdb"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:41.180970 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-64b1f39e-23de-4738-9967-5d7e4949cee0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:41.186975 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64b1f39e-23de-4738-9967-5d7e4949cee0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:41.187472 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64b1f39e-23de-4738-9967-5d7e4949cee0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:41.187901 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64b1f39e-23de-4738-9967-5d7e4949cee0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:41.188575 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64b1f39e-23de-4738-9967-5d7e4949cee0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:41.189019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64b1f39e-23de-4738-9967-5d7e4949cee0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:41.189487 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64b1f39e-23de-4738-9967-5d7e4949cee0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:41.384054 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fc1537e3-1dee-4faf-a375-94ba8dd58a65 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:41.384757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fc1537e3-1dee-4faf-a375-94ba8dd58a65 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:41.391095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc1537e3-1dee-4faf-a375-94ba8dd58a65 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:41.391728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc1537e3-1dee-4faf-a375-94ba8dd58a65 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:41.392000 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc1537e3-1dee-4faf-a375-94ba8dd58a65 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:41.392709 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc1537e3-1dee-4faf-a375-94ba8dd58a65 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:41.393186 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc1537e3-1dee-4faf-a375-94ba8dd58a65 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:41.393623 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc1537e3-1dee-4faf-a375-94ba8dd58a65 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:41.522290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-fc1537e3-1dee-4faf-a375-94ba8dd58a65 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:41.529880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-fc1537e3-1dee-4faf-a375-94ba8dd58a65 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.150308 Oct 20 09:02:41.530446 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 181/338] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:41 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:41.566716 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:41.567326 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:41.573593 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:41.574068 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:41.574529 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:41.575213 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:41.575662 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:41.576084 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:41.703054 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:41.711194 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:41.712637 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:41.713785 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-cd7da777-e94b-4f06-b523-8e63089f3152 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.150914 Oct 20 09:02:41.714348 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 182/339] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:41 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:02:41.885059 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-64b1f39e-23de-4738-9967-5d7e4949cee0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "POST /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.710928 Oct 20 09:02:41.885786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 158/340] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:02:41 2022] POST /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67/os-volume_attachments => generated 194 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:02:42.011728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4547f472-7310-475d-9dc4-9f33eaf678bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: .version_select at 0x7f5afbc2a1f0>, body: {"keypair": {"name": "tempest-keypair-528204325", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFkw8ypTj0ki2ZkrgiNJGpe4kOzujjVFz+4h9g3NV2262A6TabLHcNNngI9aJ/rUYODFBSD4kNJEA3RizqR4J+7EQjWw/F8dsAHXLQo0PvizPmUKOrjqQiPFLIylySD24w=="}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:42.027163 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-4547f472-7310-475d-9dc4-9f33eaf678bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:02:42.030845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-4547f472-7310-475d-9dc4-9f33eaf678bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for user 8002ab26f6d64822aaa65aa730781b76 and project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:02:42.051485 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4547f472-7310-475d-9dc4-9f33eaf678bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.044060 Oct 20 09:02:42.052056 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 183/341] 10.222.0.36 () {60 vars in 1159 bytes} [Thu Oct 20 09:02:42 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:02:42.564617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b515bb6b-06f3-4012-9520-e9ced8a7d7a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:42.571782 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b515bb6b-06f3-4012-9520-e9ced8a7d7a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:42.582724 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b515bb6b-06f3-4012-9520-e9ced8a7d7a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:42.583855 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b515bb6b-06f3-4012-9520-e9ced8a7d7a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:42.584718 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b515bb6b-06f3-4012-9520-e9ced8a7d7a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:42.587118 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b515bb6b-06f3-4012-9520-e9ced8a7d7a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:42.587849 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b515bb6b-06f3-4012-9520-e9ced8a7d7a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:42.588697 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b515bb6b-06f3-4012-9520-e9ced8a7d7a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:42.748249 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:42.751755 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:42.767707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:42.768575 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:42.769339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:42.771006 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:42.771840 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:42.772723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:42.980652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:42.990020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:42.991898 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:42.993323 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bc3c303b-28cd-4799-a9ac-25f093008522 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.257605 Oct 20 09:02:42.994390 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 184/342] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:42 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:43.196252 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b515bb6b-06f3-4012-9520-e9ced8a7d7a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:43.209117 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b515bb6b-06f3-4012-9520-e9ced8a7d7a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.659712 Oct 20 09:02:43.215448 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 159/343] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:42 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 667 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:44.021671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:44.022429 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:44.035761 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:44.036427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:44.036977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:44.041081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:44.041747 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:44.042536 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:44.237284 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-307549854", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b8327be3-f028-4ebc-9b25-81152867d29e"}], "security_groups": [{"name": "tempest-securitygroup--1748743475"}], "key_name": "tempest-keypair-528204325", "user_data": "}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:44.342493 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:44.356645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:44.358709 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:44.364335 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e37d50fb-470c-451f-9c3b-ad9db5727445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.346765 Oct 20 09:02:44.382560 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 185/344] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:44 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:02:44.394333 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7eb359ca-0aa6-499c-b80a-b10e73105603 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:44.395339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7eb359ca-0aa6-499c-b80a-b10e73105603 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:44.408922 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7eb359ca-0aa6-499c-b80a-b10e73105603 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:44.409732 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7eb359ca-0aa6-499c-b80a-b10e73105603 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:44.410825 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7eb359ca-0aa6-499c-b80a-b10e73105603 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:44.412068 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7eb359ca-0aa6-499c-b80a-b10e73105603 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:44.412924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7eb359ca-0aa6-499c-b80a-b10e73105603 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:44.414101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7eb359ca-0aa6-499c-b80a-b10e73105603 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:44.416668 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] validate_networks() for [('b8327be3-f028-4ebc-9b25-81152867d29e', None, None, None, None, None)] {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:02:44.605512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7eb359ca-0aa6-499c-b80a-b10e73105603 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:44.616286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7eb359ca-0aa6-499c-b80a-b10e73105603 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.232179 Oct 20 09:02:44.636013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 186/345] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:44 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:44.941341 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:02:44.941773 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:02:44.942387 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:02:44.943039 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:02:44.943457 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:02:44.953301 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:44.953789 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:44.954225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:44.954999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:44.955451 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:44.955882 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:44.974513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:02:44.978868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for user 8002ab26f6d64822aaa65aa730781b76 and project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:02:44.990245 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:02:44.992645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:02:45.042210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-eb3e2790-00c6-45c9-bd5a-084f6a5475e1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.817087 Oct 20 09:02:45.043324 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 160/346] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:02:44 2022] POST /compute/v2.1/servers => generated 402 bytes in 820 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:02:45.056196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-97ddb0c1-8939-4b57-8903-e036ebda2ee3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:45.057754 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-97ddb0c1-8939-4b57-8903-e036ebda2ee3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:45.117088 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-97ddb0c1-8939-4b57-8903-e036ebda2ee3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:45.126375 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-97ddb0c1-8939-4b57-8903-e036ebda2ee3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1240 microversion: 2.1 time: 0.074056 Oct 20 09:02:45.138540 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 187/347] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:45 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1240 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:45.396496 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:45.397102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:45.407400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:45.407992 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:45.408600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:45.409552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:45.410345 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:45.410824 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:45.585521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:45.597390 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:45.599189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:45.600568 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e5bd7f98-fd5a-43ad-b4a6-6cced3d2e9e7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.209207 Oct 20 09:02:45.601737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 161/348] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:45 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:45.645140 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c90bff71-4f9f-4863-8786-63d72507a1b9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:45.645742 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c90bff71-4f9f-4863-8786-63d72507a1b9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:45.652701 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c90bff71-4f9f-4863-8786-63d72507a1b9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:45.653183 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c90bff71-4f9f-4863-8786-63d72507a1b9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:45.653615 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c90bff71-4f9f-4863-8786-63d72507a1b9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:45.654593 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c90bff71-4f9f-4863-8786-63d72507a1b9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:45.655447 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c90bff71-4f9f-4863-8786-63d72507a1b9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:45.656107 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c90bff71-4f9f-4863-8786-63d72507a1b9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:45.833513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c90bff71-4f9f-4863-8786-63d72507a1b9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:45.842736 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c90bff71-4f9f-4863-8786-63d72507a1b9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1500 microversion: 2.1 time: 0.201148 Oct 20 09:02:45.843406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 188/349] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:45 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1500 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:45.870418 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "19b9a62b-be0e-4ba4-9b34-6d87e5cd4162"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:45.873046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:45.889508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:45.890387 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:45.891579 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:45.892384 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:45.893784 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:45.894338 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:45.921492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] HTTP exception thrown: Cannot 'attach_volume' instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 while it is in vm_state rescued Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Request method failure captured: Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: POST /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150/os-volume_attachments HTTP/1.1 Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 74 Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ44IzvbrfISoSWQdaz7A4Y4mCB866TdY__L0_HKNgMuGtSn76PJ_HBHhaSlJVS78rXHSeDqK-8OZHSlR6XDrjto3KeVysuo0baUN07Q2QroH6vEp0-TETCDD7MgP6eUx6PZ5XfPA9gmR0Yq3sbUMXAjpaWW4IJsJYamevW56vxD4ZM Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: d24a1fc0e9344af181d49cfc58f0c88b Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:02:45.923927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/d24a1fc0e9344af181d49cfc58f0c88b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_d24a1fc0e9344af181d49cfc58f0c88b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/d24a1fc0e9344af181d49cfc58f0c88b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/d24a1fc0e9344af181d49cfc58f0c88b"}]}] Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: d24a1fc0e9344af181d49cfc58f0c88b Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-ServerRescueNegativeTestJSON-1468667757-project Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 7f932695a5994c5790d14c210ea65326 Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1468667757-project Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {"volumeAttachment": {"volumeId": "19b9a62b-be0e-4ba4-9b34-6d87e5cd4162"}} Oct 20 09:02:45.925539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Cannot 'attach_volume' instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 while it is in vm_state rescued Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'server_id': '6cabc1ab-03aa-4ff2-be06-d99f9efc6150', 'body': {'volumeAttachment': {'volumeId': '19b9a62b-be0e-4ba4-9b34-6d87e5cd4162'}}} Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:02:45.926431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 while it is in vm_state rescued Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:02:45.927933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:02:45.930569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:02:45.930569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:02:45.930569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 while it is in vm_state rescued Oct 20 09:02:45.930569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:02:45.930569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Returning 409 to user: Cannot 'attach_volume' instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 while it is in vm_state rescued {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:02:45.930569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-07e4f5cf-a13e-4b75-b267-c9a49b49b97e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.062966 Oct 20 09:02:45.930569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 162/350] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:02:45 2022] POST /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150/os-volume_attachments => generated 152 bytes in 65 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 20 09:02:45.938977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2557d5d4-ff7e-4b9a-bdb0-3458e3bae51e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:45.939811 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2557d5d4-ff7e-4b9a-bdb0-3458e3bae51e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:45.946748 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2557d5d4-ff7e-4b9a-bdb0-3458e3bae51e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:45.947481 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2557d5d4-ff7e-4b9a-bdb0-3458e3bae51e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:45.948134 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2557d5d4-ff7e-4b9a-bdb0-3458e3bae51e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:45.948841 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2557d5d4-ff7e-4b9a-bdb0-3458e3bae51e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:45.949299 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2557d5d4-ff7e-4b9a-bdb0-3458e3bae51e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:45.949727 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2557d5d4-ff7e-4b9a-bdb0-3458e3bae51e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:46.034114 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2557d5d4-ff7e-4b9a-bdb0-3458e3bae51e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150/action" status: 202 len: 0 microversion: 2.1 time: 0.099224 Oct 20 09:02:46.034776 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 189/351] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:02:45 2022] POST /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150/action => generated 0 bytes in 101 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 20 09:02:46.043508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7a4f0f34-fb0a-4547-922d-ce85b441c3ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:46.044117 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7a4f0f34-fb0a-4547-922d-ce85b441c3ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:46.050684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a4f0f34-fb0a-4547-922d-ce85b441c3ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:46.051447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a4f0f34-fb0a-4547-922d-ce85b441c3ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:46.051654 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a4f0f34-fb0a-4547-922d-ce85b441c3ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:46.052285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a4f0f34-fb0a-4547-922d-ce85b441c3ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:46.052744 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a4f0f34-fb0a-4547-922d-ce85b441c3ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:46.053173 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a4f0f34-fb0a-4547-922d-ce85b441c3ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:46.157487 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f0281833-72e8-42cd-a355-99c80f471816 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:46.159143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f0281833-72e8-42cd-a355-99c80f471816 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:46.177397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0281833-72e8-42cd-a355-99c80f471816 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:46.177889 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0281833-72e8-42cd-a355-99c80f471816 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:46.180783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0281833-72e8-42cd-a355-99c80f471816 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:46.181544 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0281833-72e8-42cd-a355-99c80f471816 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:46.182013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0281833-72e8-42cd-a355-99c80f471816 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:46.182446 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0281833-72e8-42cd-a355-99c80f471816 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:46.272495 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7a4f0f34-fb0a-4547-922d-ce85b441c3ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:46.293292 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7a4f0f34-fb0a-4547-922d-ce85b441c3ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1508 microversion: 2.1 time: 0.253441 Oct 20 09:02:46.294156 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 163/352] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:46 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1508 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:46.341994 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f0281833-72e8-42cd-a355-99c80f471816 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:46.369931 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f0281833-72e8-42cd-a355-99c80f471816 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1292 microversion: 2.1 time: 0.209192 Oct 20 09:02:46.370614 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 190/353] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:46 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1292 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:46.628030 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:46.629581 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:46.636125 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:46.636686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:46.637194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:46.638104 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:46.638667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:46.639290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:46.797986 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:46.807990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:46.809570 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:46.810874 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a25ab7e0-d1e9-4549-a37a-e80b63757af8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.187149 Oct 20 09:02:46.811712 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 164/354] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:46 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:47.319444 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-447599a4-da0b-4604-b573-7daa3e0e163c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:47.321370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-447599a4-da0b-4604-b573-7daa3e0e163c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:47.329927 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-447599a4-da0b-4604-b573-7daa3e0e163c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:47.346234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-447599a4-da0b-4604-b573-7daa3e0e163c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:47.349670 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-447599a4-da0b-4604-b573-7daa3e0e163c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:47.351078 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-447599a4-da0b-4604-b573-7daa3e0e163c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:47.351783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-447599a4-da0b-4604-b573-7daa3e0e163c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:47.354099 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-447599a4-da0b-4604-b573-7daa3e0e163c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:47.396910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f86ad601-4194-44ce-a9ad-d0551559f2e9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:47.398892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f86ad601-4194-44ce-a9ad-d0551559f2e9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:47.408174 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f86ad601-4194-44ce-a9ad-d0551559f2e9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:47.408683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f86ad601-4194-44ce-a9ad-d0551559f2e9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:47.409134 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f86ad601-4194-44ce-a9ad-d0551559f2e9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:47.422444 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f86ad601-4194-44ce-a9ad-d0551559f2e9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:47.424187 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f86ad601-4194-44ce-a9ad-d0551559f2e9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:47.425382 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f86ad601-4194-44ce-a9ad-d0551559f2e9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:47.541102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f86ad601-4194-44ce-a9ad-d0551559f2e9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:47.552437 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f86ad601-4194-44ce-a9ad-d0551559f2e9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1298 microversion: 2.1 time: 0.159041 Oct 20 09:02:47.553589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 165/355] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:47 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1298 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:47.596333 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-447599a4-da0b-4604-b573-7daa3e0e163c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:47.604263 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-447599a4-da0b-4604-b573-7daa3e0e163c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1508 microversion: 2.1 time: 0.288857 Oct 20 09:02:47.619095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 191/356] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:47 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1508 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:47.841800 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:47.842409 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:47.862119 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:47.862677 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:47.863091 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:47.863801 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:47.864251 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:47.864672 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:47.871766 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:48.091757 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:48.102405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:48.104615 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:48.106910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2c5ea355-bbd2-4a32-8e6e-00ce19160027 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.268952 Oct 20 09:02:48.119697 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 166/357] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:47 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:48.577386 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5f92ccc2-9288-4268-9b7a-5f95c9c3c734 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:48.578244 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5f92ccc2-9288-4268-9b7a-5f95c9c3c734 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:48.581923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:48.585923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f92ccc2-9288-4268-9b7a-5f95c9c3c734 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:48.586400 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f92ccc2-9288-4268-9b7a-5f95c9c3c734 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:48.586862 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f92ccc2-9288-4268-9b7a-5f95c9c3c734 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:48.587574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f92ccc2-9288-4268-9b7a-5f95c9c3c734 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:48.588055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f92ccc2-9288-4268-9b7a-5f95c9c3c734 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:48.588634 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f92ccc2-9288-4268-9b7a-5f95c9c3c734 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:48.633258 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9f21d063-b13b-4f1f-9c43-1f796f2a5833 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:48.634011 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-9f21d063-b13b-4f1f-9c43-1f796f2a5833 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:48.640197 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f21d063-b13b-4f1f-9c43-1f796f2a5833 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:48.640936 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f21d063-b13b-4f1f-9c43-1f796f2a5833 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:48.641533 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f21d063-b13b-4f1f-9c43-1f796f2a5833 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:48.642531 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f21d063-b13b-4f1f-9c43-1f796f2a5833 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:48.643207 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f21d063-b13b-4f1f-9c43-1f796f2a5833 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:48.644115 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f21d063-b13b-4f1f-9c43-1f796f2a5833 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:48.792406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5f92ccc2-9288-4268-9b7a-5f95c9c3c734 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:48.813117 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5f92ccc2-9288-4268-9b7a-5f95c9c3c734 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.239422 Oct 20 09:02:48.813809 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 192/358] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:48 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:48.836759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-9f21d063-b13b-4f1f-9c43-1f796f2a5833 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:48.845221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-9f21d063-b13b-4f1f-9c43-1f796f2a5833 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1508 microversion: 2.1 time: 0.215858 Oct 20 09:02:48.846545 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 167/359] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:48 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1508 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:49.135465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:49.136080 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:49.144115 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:49.144645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:49.145082 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:49.159176 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:49.159687 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:49.160144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:49.312424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:49.322253 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:49.331288 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:49.332710 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-258ddc64-6cc2-4f0f-8356-2811f7c686f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.201071 Oct 20 09:02:49.333594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 193/360] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:49 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:49.839430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-718830ba-41c9-4455-b799-a715642f8fb5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:49.840431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-718830ba-41c9-4455-b799-a715642f8fb5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:49.846931 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-718830ba-41c9-4455-b799-a715642f8fb5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:49.848125 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-718830ba-41c9-4455-b799-a715642f8fb5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:49.848737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-718830ba-41c9-4455-b799-a715642f8fb5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:49.849702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-718830ba-41c9-4455-b799-a715642f8fb5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:49.850153 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-718830ba-41c9-4455-b799-a715642f8fb5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:49.851359 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-718830ba-41c9-4455-b799-a715642f8fb5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:49.869974 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b453907b-a8f7-4adf-8ed4-e1742fba90e5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:49.870718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b453907b-a8f7-4adf-8ed4-e1742fba90e5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:49.884369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b453907b-a8f7-4adf-8ed4-e1742fba90e5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:49.885971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b453907b-a8f7-4adf-8ed4-e1742fba90e5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:49.887632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b453907b-a8f7-4adf-8ed4-e1742fba90e5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:49.888911 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b453907b-a8f7-4adf-8ed4-e1742fba90e5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:49.889769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b453907b-a8f7-4adf-8ed4-e1742fba90e5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:49.891172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b453907b-a8f7-4adf-8ed4-e1742fba90e5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:50.106823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-718830ba-41c9-4455-b799-a715642f8fb5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:50.115638 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-718830ba-41c9-4455-b799-a715642f8fb5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.280485 Oct 20 09:02:50.120525 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 168/361] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:49 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:50.153315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b453907b-a8f7-4adf-8ed4-e1742fba90e5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:50.164625 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b453907b-a8f7-4adf-8ed4-e1742fba90e5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1508 microversion: 2.1 time: 0.296793 Oct 20 09:02:50.165440 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 194/362] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:49 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1508 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:50.359948 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:50.360594 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:50.367780 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:50.368533 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:50.369120 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:50.369989 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:50.370634 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:50.371208 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:50.584906 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:50.595016 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:50.597919 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:50.601641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-55fec1d4-342a-44f1-ac00-100767be7fe4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.243432 Oct 20 09:02:50.616850 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 169/363] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:50 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:50.894705 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-d6d4bb0e-79af-49d5-99db-89a7f077966d req-3abf7d9d-2229-4558-a4b8-fc68388fee50 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aae7b08a-4c9a-4d12-9598-193afab4ca00", "tag": "68cd5f68-1310-4334-b214-dc31475d3f8f"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:50.906388 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-d6d4bb0e-79af-49d5-99db-89a7f077966d req-3abf7d9d-2229-4558-a4b8-fc68388fee50 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:50.907261 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-d6d4bb0e-79af-49d5-99db-89a7f077966d req-3abf7d9d-2229-4558-a4b8-fc68388fee50 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:50.907919 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-d6d4bb0e-79af-49d5-99db-89a7f077966d req-3abf7d9d-2229-4558-a4b8-fc68388fee50 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:50.920265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-d6d4bb0e-79af-49d5-99db-89a7f077966d req-3abf7d9d-2229-4558-a4b8-fc68388fee50 service nova] Creating event network-changed:68cd5f68-1310-4334-b214-dc31475d3f8f for instance aae7b08a-4c9a-4d12-9598-193afab4ca00 on hv-cinder-maste Oct 20 09:02:50.925936 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-d6d4bb0e-79af-49d5-99db-89a7f077966d req-3abf7d9d-2229-4558-a4b8-fc68388fee50 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034969 Oct 20 09:02:50.926926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 195/364] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:02:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:02:51.141738 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-90dbbd6f-2739-4e71-9887-9c2a4e12a452 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:51.142253 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-90dbbd6f-2739-4e71-9887-9c2a4e12a452 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:51.149294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90dbbd6f-2739-4e71-9887-9c2a4e12a452 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:51.150248 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90dbbd6f-2739-4e71-9887-9c2a4e12a452 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:51.150652 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90dbbd6f-2739-4e71-9887-9c2a4e12a452 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:51.151379 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90dbbd6f-2739-4e71-9887-9c2a4e12a452 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:51.151823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90dbbd6f-2739-4e71-9887-9c2a4e12a452 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:51.152584 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90dbbd6f-2739-4e71-9887-9c2a4e12a452 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:51.189822 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-20837e1c-e99a-4503-aa7c-51273180e3ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:51.190540 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-20837e1c-e99a-4503-aa7c-51273180e3ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:51.197600 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-20837e1c-e99a-4503-aa7c-51273180e3ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:51.198414 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-20837e1c-e99a-4503-aa7c-51273180e3ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:51.199148 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-20837e1c-e99a-4503-aa7c-51273180e3ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:51.200053 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-20837e1c-e99a-4503-aa7c-51273180e3ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:51.200522 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-20837e1c-e99a-4503-aa7c-51273180e3ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:51.200953 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-20837e1c-e99a-4503-aa7c-51273180e3ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:51.337884 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-90dbbd6f-2739-4e71-9887-9c2a4e12a452 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:51.347520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-90dbbd6f-2739-4e71-9887-9c2a4e12a452 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.209762 Oct 20 09:02:51.348110 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 170/365] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:51 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:51.384944 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-20837e1c-e99a-4503-aa7c-51273180e3ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:51.398873 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-20837e1c-e99a-4503-aa7c-51273180e3ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1514 microversion: 2.1 time: 0.212297 Oct 20 09:02:51.399787 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 196/366] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:51 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1514 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:02:51.628422 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:51.629027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:51.635982 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:51.637939 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:51.638392 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:51.639128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:51.639591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:51.640017 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:51.855550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:51.864552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:51.868008 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:51.869459 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d05bfbc2-6d44-4d7f-9596-0802eeea057c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.244978 Oct 20 09:02:51.881123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 171/367] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:51 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:52.377468 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d7b4e06f-40cd-451c-800d-d683cf4d12d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:52.378076 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d7b4e06f-40cd-451c-800d-d683cf4d12d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:52.385525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d7b4e06f-40cd-451c-800d-d683cf4d12d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:52.386016 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d7b4e06f-40cd-451c-800d-d683cf4d12d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:52.386449 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d7b4e06f-40cd-451c-800d-d683cf4d12d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:52.387196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d7b4e06f-40cd-451c-800d-d683cf4d12d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:52.387645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d7b4e06f-40cd-451c-800d-d683cf4d12d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:52.388066 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d7b4e06f-40cd-451c-800d-d683cf4d12d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:52.545235 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d7b4e06f-40cd-451c-800d-d683cf4d12d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:52.549385 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:52.556017 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d7b4e06f-40cd-451c-800d-d683cf4d12d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.182474 Oct 20 09:02:52.571267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 197/368] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:52 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:52.913164 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:52.914345 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:52.935168 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:52.936550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:52.937815 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:52.939357 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:52.942186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:52.944454 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:53.028635 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b8d0742f-c498-4bf0-a2d7-2bc91883ff70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'create', calling method: .version_select at 0x7f5afbc1ae50>, body: {"keypair": {"name": "tempest-keypair-1923229509", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHYhmQgaLTq2Prz92Dy7rhNjaR56OhHZGLhwZCzew7XBeM8p53Iy6TkHC96oZwHwfacld9tqUNmyPOHbfqY0YSi/plE59tYsKFWhwypoE+OtQaf9qjYVwPkkYC28EQZCmA=="}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:53.050701 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-b8d0742f-c498-4bf0-a2d7-2bc91883ff70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Getting quotas for project d24a1fc0e9344af181d49cfc58f0c88b. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:02:53.055871 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-b8d0742f-c498-4bf0-a2d7-2bc91883ff70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Getting quotas for user 7f932695a5994c5790d14c210ea65326 and project d24a1fc0e9344af181d49cfc58f0c88b. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:02:53.096843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b8d0742f-c498-4bf0-a2d7-2bc91883ff70 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.074413 Oct 20 09:02:53.105389 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 198/369] 10.222.0.36 () {60 vars in 1159 bytes} [Thu Oct 20 09:02:53 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:02:53.151071 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:53.173657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:53.181219 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:53.182536 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f7026834-06f3-4b83-ba0a-888c2288c51e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.280078 Oct 20 09:02:53.183133 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 172/370] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:52 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:02:53.584871 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-db930936-7fa4-41c7-832a-3e5d57de68f6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:53.587517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-db930936-7fa4-41c7-832a-3e5d57de68f6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:53.595644 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db930936-7fa4-41c7-832a-3e5d57de68f6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:53.596550 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db930936-7fa4-41c7-832a-3e5d57de68f6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:53.597831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db930936-7fa4-41c7-832a-3e5d57de68f6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:53.599681 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db930936-7fa4-41c7-832a-3e5d57de68f6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:53.601739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db930936-7fa4-41c7-832a-3e5d57de68f6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:53.603367 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db930936-7fa4-41c7-832a-3e5d57de68f6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:53.802077 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-db930936-7fa4-41c7-832a-3e5d57de68f6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:53.817697 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-db930936-7fa4-41c7-832a-3e5d57de68f6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.239128 Oct 20 09:02:53.818977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 199/371] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:53 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:54.210664 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:54.212275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:54.221554 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:54.222595 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:54.223400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:54.224580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:54.225248 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:54.225987 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:54.464711 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:54.474508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:54.476980 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:54.478272 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-18f64003-5983-4119-9209-030ded92c01d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.272789 Oct 20 09:02:54.478925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 173/372] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:54 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:54.846007 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-050ba5e4-1771-4c07-830f-6fee05e26aa8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:54.847479 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-050ba5e4-1771-4c07-830f-6fee05e26aa8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:54.864756 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-050ba5e4-1771-4c07-830f-6fee05e26aa8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:54.865333 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-050ba5e4-1771-4c07-830f-6fee05e26aa8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:54.865846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-050ba5e4-1771-4c07-830f-6fee05e26aa8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:54.867291 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-050ba5e4-1771-4c07-830f-6fee05e26aa8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:54.867812 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-050ba5e4-1771-4c07-830f-6fee05e26aa8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:54.868956 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-050ba5e4-1771-4c07-830f-6fee05e26aa8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:55.012156 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-245615168", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "89681d33-1e95-46d3-a4f1-4813c7b75e62"}], "security_groups": [{"name": "tempest-securitygroup--335886864"}], "key_name": "tempest-keypair-1923229509", "user_data": "}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:02:55.023478 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:55.024580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=6 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:02:55.027727 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-050ba5e4-1771-4c07-830f-6fee05e26aa8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:55.036590 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-050ba5e4-1771-4c07-830f-6fee05e26aa8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.194466 Oct 20 09:02:55.039214 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 200/373] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:54 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:55.139302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] validate_networks() for [('89681d33-1e95-46d3-a4f1-4813c7b75e62', None, None, None, None, None)] {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:02:55.512681 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:55.513596 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:55.521723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:55.523118 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:55.523741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:55.524606 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:55.525214 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:55.526960 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:55.589289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:02:55.591405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:02:55.592083 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:02:55.592491 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:02:55.592951 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:02:55.608634 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:55.609949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:55.610400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:55.611455 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:55.613246 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:55.614223 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:55.634247 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Getting quotas for project d24a1fc0e9344af181d49cfc58f0c88b. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:02:55.639710 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Getting quotas for user 7f932695a5994c5790d14c210ea65326 and project d24a1fc0e9344af181d49cfc58f0c88b. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:02:55.652967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:02:55.655457 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:02:55.681356 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:55.689927 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:55.692395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:55.693632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1b7539d3-9f22-43dc-b5d5-ba8dcb2dd6d4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.190295 Oct 20 09:02:55.697314 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 201/374] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:55 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:02:55.718541 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4479f286-0aa2-4118-873b-adb72f90ac3c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.712556 Oct 20 09:02:55.719614 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 174/375] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:02:55 2022] POST /compute/v2.1/servers => generated 408 bytes in 714 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:02:55.730380 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-63236aac-29ca-4c47-a389-afe9d3f83745 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:55.731399 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-63236aac-29ca-4c47-a389-afe9d3f83745 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:55.822687 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-63236aac-29ca-4c47-a389-afe9d3f83745 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:55.830399 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-63236aac-29ca-4c47-a389-afe9d3f83745 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1245 microversion: 2.1 time: 0.103139 Oct 20 09:02:55.839616 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 202/376] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:55 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1245 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:56.068055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cd4623fa-d696-47a8-8432-c07b9708fc9c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:56.068055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-cd4623fa-d696-47a8-8432-c07b9708fc9c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:56.077406 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cd4623fa-d696-47a8-8432-c07b9708fc9c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:56.080944 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cd4623fa-d696-47a8-8432-c07b9708fc9c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:56.083147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cd4623fa-d696-47a8-8432-c07b9708fc9c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:56.084576 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cd4623fa-d696-47a8-8432-c07b9708fc9c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:56.086514 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cd4623fa-d696-47a8-8432-c07b9708fc9c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:56.087915 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cd4623fa-d696-47a8-8432-c07b9708fc9c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:56.247276 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-cd4623fa-d696-47a8-8432-c07b9708fc9c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:56.255672 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-cd4623fa-d696-47a8-8432-c07b9708fc9c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.198117 Oct 20 09:02:56.266771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 175/377] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:56 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:56.720186 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:56.720841 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:56.727981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:56.728666 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:56.729262 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:56.730190 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:56.730856 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:56.731476 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:56.855138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0cbbc073-3204-46c8-86f3-abbf7c29b0a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:56.856866 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0cbbc073-3204-46c8-86f3-abbf7c29b0a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:56.865523 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0cbbc073-3204-46c8-86f3-abbf7c29b0a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:56.866130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0cbbc073-3204-46c8-86f3-abbf7c29b0a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:56.866692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0cbbc073-3204-46c8-86f3-abbf7c29b0a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:56.867917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0cbbc073-3204-46c8-86f3-abbf7c29b0a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:56.868600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0cbbc073-3204-46c8-86f3-abbf7c29b0a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:56.869085 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0cbbc073-3204-46c8-86f3-abbf7c29b0a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:56.889277 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:56.902166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:56.904649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:56.906702 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-52564883-9d09-4db1-a5b8-3094a43270e3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.191357 Oct 20 09:02:56.908064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 203/378] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:56 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:56.972810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0cbbc073-3204-46c8-86f3-abbf7c29b0a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:56.981410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0cbbc073-3204-46c8-86f3-abbf7c29b0a3 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1249 microversion: 2.1 time: 0.130332 Oct 20 09:02:56.982100 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 176/379] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:56 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1249 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:57.281269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0eda0c95-509f-44ea-acb0-bf9b71614df4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:57.281894 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0eda0c95-509f-44ea-acb0-bf9b71614df4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:57.288515 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0eda0c95-509f-44ea-acb0-bf9b71614df4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:57.289034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0eda0c95-509f-44ea-acb0-bf9b71614df4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:57.289471 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0eda0c95-509f-44ea-acb0-bf9b71614df4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:57.290200 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0eda0c95-509f-44ea-acb0-bf9b71614df4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:57.290833 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0eda0c95-509f-44ea-acb0-bf9b71614df4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:57.291421 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0eda0c95-509f-44ea-acb0-bf9b71614df4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:57.525075 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0eda0c95-509f-44ea-acb0-bf9b71614df4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:57.537265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0eda0c95-509f-44ea-acb0-bf9b71614df4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.259503 Oct 20 09:02:57.550561 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 204/380] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:57 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:57.933772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:57.934379 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:57.943166 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:57.943657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:57.944091 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:57.945424 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:57.945877 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:57.946298 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:58.004629 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b817e26e-d85b-474f-af84-de0c5440487f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:58.005952 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b817e26e-d85b-474f-af84-de0c5440487f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:58.013673 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b817e26e-d85b-474f-af84-de0c5440487f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:58.014637 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b817e26e-d85b-474f-af84-de0c5440487f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:58.015350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b817e26e-d85b-474f-af84-de0c5440487f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:58.016258 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b817e26e-d85b-474f-af84-de0c5440487f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:58.017156 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b817e26e-d85b-474f-af84-de0c5440487f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:58.017795 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b817e26e-d85b-474f-af84-de0c5440487f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:58.099528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b817e26e-d85b-474f-af84-de0c5440487f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:58.109204 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b817e26e-d85b-474f-af84-de0c5440487f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1297 microversion: 2.1 time: 0.108609 Oct 20 09:02:58.109786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 205/381] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:57 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1297 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:58.136422 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:58.145430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:58.147818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:58.149203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-cb2935b2-bf76-45d1-87a7-c344f4d23662 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1741 microversion: 2.20 time: 0.219360 Oct 20 09:02:58.150109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 177/382] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:57 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1741 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:02:58.565458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-149ccb49-ddb5-45cb-a675-88e362c51830 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:58.566131 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-149ccb49-ddb5-45cb-a675-88e362c51830 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:58.572485 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-149ccb49-ddb5-45cb-a675-88e362c51830 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:58.572992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-149ccb49-ddb5-45cb-a675-88e362c51830 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:58.573444 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-149ccb49-ddb5-45cb-a675-88e362c51830 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:58.574149 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-149ccb49-ddb5-45cb-a675-88e362c51830 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:58.574623 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-149ccb49-ddb5-45cb-a675-88e362c51830 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:58.575049 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-149ccb49-ddb5-45cb-a675-88e362c51830 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:58.727407 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-149ccb49-ddb5-45cb-a675-88e362c51830 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:58.742073 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-149ccb49-ddb5-45cb-a675-88e362c51830 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.180401 Oct 20 09:02:58.743207 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 206/383] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:58 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:59.139660 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6df6feff-2cd7-449e-bad3-787a6bb3e7f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:59.140284 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6df6feff-2cd7-449e-bad3-787a6bb3e7f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:59.150505 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6df6feff-2cd7-449e-bad3-787a6bb3e7f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:59.152153 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6df6feff-2cd7-449e-bad3-787a6bb3e7f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:59.152679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6df6feff-2cd7-449e-bad3-787a6bb3e7f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:59.153550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6df6feff-2cd7-449e-bad3-787a6bb3e7f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:59.154403 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6df6feff-2cd7-449e-bad3-787a6bb3e7f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:59.155134 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6df6feff-2cd7-449e-bad3-787a6bb3e7f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:59.175461 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:59.176796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:59.185189 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:59.186203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:59.187037 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:59.188198 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:59.188880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:59.189623 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:59.234842 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6df6feff-2cd7-449e-bad3-787a6bb3e7f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:59.250645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6df6feff-2cd7-449e-bad3-787a6bb3e7f7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1297 microversion: 2.1 time: 0.116133 Oct 20 09:02:59.252153 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 178/384] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:59 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1297 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:02:59.354086 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:59.363853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:59.365632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:59.367534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-48bfb28b-d020-4ca2-ba10-cf0642d1cb6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1741 microversion: 2.20 time: 0.196500 Oct 20 09:02:59.382993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 207/385] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:02:59 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1741 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:02:59.765461 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-399991a4-1cf4-4796-8163-d2273f4c6e87 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:02:59.766194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-399991a4-1cf4-4796-8163-d2273f4c6e87 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:02:59.776800 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-399991a4-1cf4-4796-8163-d2273f4c6e87 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:59.777278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-399991a4-1cf4-4796-8163-d2273f4c6e87 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:59.777620 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-399991a4-1cf4-4796-8163-d2273f4c6e87 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:59.778306 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-399991a4-1cf4-4796-8163-d2273f4c6e87 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:02:59.778776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-399991a4-1cf4-4796-8163-d2273f4c6e87 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:02:59.779194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-399991a4-1cf4-4796-8163-d2273f4c6e87 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:02:59.913552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-399991a4-1cf4-4796-8163-d2273f4c6e87 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:02:59.921205 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-399991a4-1cf4-4796-8163-d2273f4c6e87 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.159837 Oct 20 09:02:59.921764 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 179/386] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:02:59 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:00.278588 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5bd0d3ac-fa43-4182-8ae1-aefeae05aca2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:00.279233 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5bd0d3ac-fa43-4182-8ae1-aefeae05aca2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:00.285519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bd0d3ac-fa43-4182-8ae1-aefeae05aca2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:00.285997 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bd0d3ac-fa43-4182-8ae1-aefeae05aca2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:00.286427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bd0d3ac-fa43-4182-8ae1-aefeae05aca2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:00.287135 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bd0d3ac-fa43-4182-8ae1-aefeae05aca2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:00.287581 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bd0d3ac-fa43-4182-8ae1-aefeae05aca2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:00.288029 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bd0d3ac-fa43-4182-8ae1-aefeae05aca2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:00.348569 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5bd0d3ac-fa43-4182-8ae1-aefeae05aca2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:00.356093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5bd0d3ac-fa43-4182-8ae1-aefeae05aca2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1297 microversion: 2.1 time: 0.081403 Oct 20 09:03:00.356657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 208/387] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:00 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1297 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:00.393116 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:00.393720 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:00.399675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:00.400176 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:00.400614 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:00.401291 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:00.401737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:00.402157 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:00.526870 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:00.534832 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:00.536275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:00.537493 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-852993ad-e64f-4d62-b161-fd9aeca71be1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1741 microversion: 2.20 time: 0.148172 Oct 20 09:03:00.538588 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 180/388] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:00 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1741 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:00.943455 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-effcc6b5-2bab-4e3e-8720-4b312256d4f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:00.944064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-effcc6b5-2bab-4e3e-8720-4b312256d4f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:00.952332 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-effcc6b5-2bab-4e3e-8720-4b312256d4f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:00.953108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-effcc6b5-2bab-4e3e-8720-4b312256d4f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:00.953679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-effcc6b5-2bab-4e3e-8720-4b312256d4f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:00.954562 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-effcc6b5-2bab-4e3e-8720-4b312256d4f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:00.955143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-effcc6b5-2bab-4e3e-8720-4b312256d4f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:00.955892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-effcc6b5-2bab-4e3e-8720-4b312256d4f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:01.344557 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-effcc6b5-2bab-4e3e-8720-4b312256d4f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:01.356070 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-effcc6b5-2bab-4e3e-8720-4b312256d4f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.416340 Oct 20 09:03:01.356643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 209/389] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:00 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:01.378161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-771c5baf-da8c-42d0-adec-c89bc5c8c1a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:01.378803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-771c5baf-da8c-42d0-adec-c89bc5c8c1a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:01.385918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-771c5baf-da8c-42d0-adec-c89bc5c8c1a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:01.387198 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-771c5baf-da8c-42d0-adec-c89bc5c8c1a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:01.387747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-771c5baf-da8c-42d0-adec-c89bc5c8c1a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:01.388783 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-771c5baf-da8c-42d0-adec-c89bc5c8c1a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:01.389240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-771c5baf-da8c-42d0-adec-c89bc5c8c1a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:01.390340 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-771c5baf-da8c-42d0-adec-c89bc5c8c1a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:01.473680 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-771c5baf-da8c-42d0-adec-c89bc5c8c1a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:01.482949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-771c5baf-da8c-42d0-adec-c89bc5c8c1a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1303 microversion: 2.1 time: 0.108576 Oct 20 09:03:01.483594 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 181/390] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:01 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1303 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:01.563894 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:01.564351 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:01.570587 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:01.571072 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:01.571503 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:01.572438 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:01.572856 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:01.573275 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:01.703782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:01.712821 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:01.714323 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:01.715764 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1a416cad-78e2-4129-8f34-6994565f658a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1741 microversion: 2.20 time: 0.155935 Oct 20 09:03:01.716657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 210/391] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:01 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1741 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:02.379474 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a35153d6-a91f-4b2c-8b40-4f5af213284e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:02.380349 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a35153d6-a91f-4b2c-8b40-4f5af213284e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:02.388542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:02.391021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a35153d6-a91f-4b2c-8b40-4f5af213284e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:02.391706 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a35153d6-a91f-4b2c-8b40-4f5af213284e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:02.392359 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a35153d6-a91f-4b2c-8b40-4f5af213284e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:02.393246 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a35153d6-a91f-4b2c-8b40-4f5af213284e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:02.394028 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a35153d6-a91f-4b2c-8b40-4f5af213284e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:02.394776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a35153d6-a91f-4b2c-8b40-4f5af213284e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:02.507549 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5b182de1-a1c9-4cf7-a92c-f62bd1c84183 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:02.508165 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5b182de1-a1c9-4cf7-a92c-f62bd1c84183 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:02.515872 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b182de1-a1c9-4cf7-a92c-f62bd1c84183 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:02.516615 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b182de1-a1c9-4cf7-a92c-f62bd1c84183 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:02.517145 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b182de1-a1c9-4cf7-a92c-f62bd1c84183 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:02.518722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b182de1-a1c9-4cf7-a92c-f62bd1c84183 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:02.518896 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b182de1-a1c9-4cf7-a92c-f62bd1c84183 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:02.519406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b182de1-a1c9-4cf7-a92c-f62bd1c84183 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:02.619986 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a35153d6-a91f-4b2c-8b40-4f5af213284e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:02.630072 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a35153d6-a91f-4b2c-8b40-4f5af213284e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.255314 Oct 20 09:03:02.645757 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 182/392] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:02 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:02.677596 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5b182de1-a1c9-4cf7-a92c-f62bd1c84183 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:02.686990 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5b182de1-a1c9-4cf7-a92c-f62bd1c84183 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1370 microversion: 2.1 time: 0.183883 Oct 20 09:03:02.687573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 211/393] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:02 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1370 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:02.744397 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:02.745182 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:02.752777 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:02.753391 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:02.753912 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:02.755129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:02.755724 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:02.756227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:02.778875 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:02.798033 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-cafc486f-bfdc-4b3e-b00f-4e5010010c49 req-a260bfc5-d24b-4775-b2bb-3f8da99c6d13 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ea0966d3-1311-4690-a69f-d3df935a76e1", "tag": "b9bd9664-f1f8-415b-84b5-9c0f08f3fc00"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:02.809551 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-cafc486f-bfdc-4b3e-b00f-4e5010010c49 req-a260bfc5-d24b-4775-b2bb-3f8da99c6d13 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:02.810034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-cafc486f-bfdc-4b3e-b00f-4e5010010c49 req-a260bfc5-d24b-4775-b2bb-3f8da99c6d13 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:02.810463 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-cafc486f-bfdc-4b3e-b00f-4e5010010c49 req-a260bfc5-d24b-4775-b2bb-3f8da99c6d13 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:02.825408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-cafc486f-bfdc-4b3e-b00f-4e5010010c49 req-a260bfc5-d24b-4775-b2bb-3f8da99c6d13 service nova] Creating event network-changed:b9bd9664-f1f8-415b-84b5-9c0f08f3fc00 for instance ea0966d3-1311-4690-a69f-d3df935a76e1 on hv-cinder-maste Oct 20 09:03:02.833075 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-cafc486f-bfdc-4b3e-b00f-4e5010010c49 req-a260bfc5-d24b-4775-b2bb-3f8da99c6d13 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038417 Oct 20 09:03:02.833556 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 212/394] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:03:02.919460 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:02.927364 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:02.928810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:02.929953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3c9579b9-e0a1-424f-9b4d-310ef687df99 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1741 microversion: 2.20 time: 0.191984 Oct 20 09:03:02.930517 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 183/395] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:02 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1741 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:03.652868 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3a8e3a87-2d67-4548-afa6-31761c7c098a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:03.653497 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3a8e3a87-2d67-4548-afa6-31761c7c098a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:03.660304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a8e3a87-2d67-4548-afa6-31761c7c098a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:03.660773 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a8e3a87-2d67-4548-afa6-31761c7c098a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:03.661195 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a8e3a87-2d67-4548-afa6-31761c7c098a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:03.661868 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a8e3a87-2d67-4548-afa6-31761c7c098a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:03.662308 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a8e3a87-2d67-4548-afa6-31761c7c098a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:03.663274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a8e3a87-2d67-4548-afa6-31761c7c098a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:03.710320 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5fa06749-8531-4426-a513-d556d8a28508 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:03.710951 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5fa06749-8531-4426-a513-d556d8a28508 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:03.717568 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5fa06749-8531-4426-a513-d556d8a28508 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:03.718050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5fa06749-8531-4426-a513-d556d8a28508 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:03.718509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5fa06749-8531-4426-a513-d556d8a28508 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:03.719195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5fa06749-8531-4426-a513-d556d8a28508 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:03.719645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5fa06749-8531-4426-a513-d556d8a28508 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:03.720077 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5fa06749-8531-4426-a513-d556d8a28508 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:03.790322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3a8e3a87-2d67-4548-afa6-31761c7c098a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:03.797873 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3a8e3a87-2d67-4548-afa6-31761c7c098a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.149182 Oct 20 09:03:03.798434 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 213/396] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:03 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:03.856982 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5fa06749-8531-4426-a513-d556d8a28508 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:03.864491 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5fa06749-8531-4426-a513-d556d8a28508 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1370 microversion: 2.1 time: 0.158013 Oct 20 09:03:03.865046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 184/397] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:03 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1370 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:03.955050 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:03.955642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:03.962069 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:03.962573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:03.963003 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:03.963673 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:03.964114 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:03.964530 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:04.092892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:04.101199 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:04.102707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:04.104021 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-aa3b242b-dc28-4a3b-a38f-39ace63a5733 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1741 microversion: 2.20 time: 0.152650 Oct 20 09:03:04.104586 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 214/398] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:03 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1741 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:04.820351 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e33f843f-63b9-4c1c-8cde-33cbef53906b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:04.820965 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e33f843f-63b9-4c1c-8cde-33cbef53906b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:04.827424 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e33f843f-63b9-4c1c-8cde-33cbef53906b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:04.827910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e33f843f-63b9-4c1c-8cde-33cbef53906b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:04.828347 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e33f843f-63b9-4c1c-8cde-33cbef53906b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:04.829031 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e33f843f-63b9-4c1c-8cde-33cbef53906b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:04.829483 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e33f843f-63b9-4c1c-8cde-33cbef53906b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:04.829914 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e33f843f-63b9-4c1c-8cde-33cbef53906b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:04.887429 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4e6a3ec5-1d37-4491-8571-565bfcbffb9e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:04.888155 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4e6a3ec5-1d37-4491-8571-565bfcbffb9e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:04.894734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e6a3ec5-1d37-4491-8571-565bfcbffb9e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:04.895294 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e6a3ec5-1d37-4491-8571-565bfcbffb9e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:04.895735 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e6a3ec5-1d37-4491-8571-565bfcbffb9e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:04.896599 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e6a3ec5-1d37-4491-8571-565bfcbffb9e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:04.897566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e6a3ec5-1d37-4491-8571-565bfcbffb9e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:04.897996 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e6a3ec5-1d37-4491-8571-565bfcbffb9e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:04.961195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e33f843f-63b9-4c1c-8cde-33cbef53906b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:04.970195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e33f843f-63b9-4c1c-8cde-33cbef53906b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.153721 Oct 20 09:03:04.970919 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 185/399] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:04 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:05.031530 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4e6a3ec5-1d37-4491-8571-565bfcbffb9e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:05.039796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4e6a3ec5-1d37-4491-8571-565bfcbffb9e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1370 microversion: 2.1 time: 0.156193 Oct 20 09:03:05.040356 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 215/400] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:04 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1370 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:05.129688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:05.130295 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:05.136815 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:05.137297 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:05.137733 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:05.138415 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:05.138892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:05.139316 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:05.271066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:05.279381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:05.280813 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:05.281942 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8e77b3bd-3d12-4f76-ba4c-b012e7e3776d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1741 microversion: 2.20 time: 0.156091 Oct 20 09:03:05.282513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 186/401] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:05 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1741 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:05.992996 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2258a4ec-29f1-4cf3-a288-e3dafe6ab685 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:05.993595 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2258a4ec-29f1-4cf3-a288-e3dafe6ab685 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:05.995563 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=12 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:06.001329 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2258a4ec-29f1-4cf3-a288-e3dafe6ab685 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:06.001808 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2258a4ec-29f1-4cf3-a288-e3dafe6ab685 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:06.002240 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2258a4ec-29f1-4cf3-a288-e3dafe6ab685 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:06.002953 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2258a4ec-29f1-4cf3-a288-e3dafe6ab685 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:06.003403 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2258a4ec-29f1-4cf3-a288-e3dafe6ab685 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:06.004022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2258a4ec-29f1-4cf3-a288-e3dafe6ab685 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:06.061801 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7d1b9abb-f8d3-47f2-83ea-1b9d89ca0148 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:06.062405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7d1b9abb-f8d3-47f2-83ea-1b9d89ca0148 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:06.064632 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:06.065447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=6 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:06.072694 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7d1b9abb-f8d3-47f2-83ea-1b9d89ca0148 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:06.073333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7d1b9abb-f8d3-47f2-83ea-1b9d89ca0148 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:06.073774 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7d1b9abb-f8d3-47f2-83ea-1b9d89ca0148 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:06.074502 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7d1b9abb-f8d3-47f2-83ea-1b9d89ca0148 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:06.074946 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7d1b9abb-f8d3-47f2-83ea-1b9d89ca0148 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:06.075368 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7d1b9abb-f8d3-47f2-83ea-1b9d89ca0148 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:06.142634 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2258a4ec-29f1-4cf3-a288-e3dafe6ab685 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:06.152224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2258a4ec-29f1-4cf3-a288-e3dafe6ab685 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.163301 Oct 20 09:03:06.156276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 216/402] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:05 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:06.272281 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7d1b9abb-f8d3-47f2-83ea-1b9d89ca0148 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:06.280435 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7d1b9abb-f8d3-47f2-83ea-1b9d89ca0148 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1370 microversion: 2.1 time: 0.222533 Oct 20 09:03:06.280995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 187/403] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:06 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1370 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:06.307016 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:06.307702 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:06.313746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:06.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:06.314731 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:06.315429 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:06.315983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:06.316416 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:06.320592 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=100 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:06.464388 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:06.473070 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:06.474560 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:06.475728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b4d419d8-b146-411c-b09a-7a2ff0cc6df8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1741 microversion: 2.20 time: 0.172601 Oct 20 09:03:06.477049 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 217/404] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:06 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1741 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:06.594376 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-5043c218-5bc6-46c3-9d86-22b4fe8d3c0b req-ccca98b8-5def-40ca-a855-93e811b7a2c0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f8d61907-24ca-4e74-917e-d218d0f89557", "name": "network-vif-plugged", "status": "completed", "tag": "6ba2009e-08be-4c65-a1d5-5e9548c8d407"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:06.604848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-5043c218-5bc6-46c3-9d86-22b4fe8d3c0b req-ccca98b8-5def-40ca-a855-93e811b7a2c0 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:06.605866 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-5043c218-5bc6-46c3-9d86-22b4fe8d3c0b req-ccca98b8-5def-40ca-a855-93e811b7a2c0 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:06.606437 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-5043c218-5bc6-46c3-9d86-22b4fe8d3c0b req-ccca98b8-5def-40ca-a855-93e811b7a2c0 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:06.622228 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-5043c218-5bc6-46c3-9d86-22b4fe8d3c0b req-ccca98b8-5def-40ca-a855-93e811b7a2c0 service nova] Creating event network-vif-plugged:6ba2009e-08be-4c65-a1d5-5e9548c8d407 for instance f8d61907-24ca-4e74-917e-d218d0f89557 on hv-cinder-maste Oct 20 09:03:06.630893 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-5043c218-5bc6-46c3-9d86-22b4fe8d3c0b req-ccca98b8-5def-40ca-a855-93e811b7a2c0 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047444 Oct 20 09:03:06.633743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 188/405] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:03:07.174900 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4633c864-c35d-41ce-a81d-c59a1305a71d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:07.176256 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4633c864-c35d-41ce-a81d-c59a1305a71d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:07.183697 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4633c864-c35d-41ce-a81d-c59a1305a71d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:07.184386 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4633c864-c35d-41ce-a81d-c59a1305a71d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:07.184979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4633c864-c35d-41ce-a81d-c59a1305a71d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:07.186003 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4633c864-c35d-41ce-a81d-c59a1305a71d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:07.186636 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4633c864-c35d-41ce-a81d-c59a1305a71d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:07.187323 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4633c864-c35d-41ce-a81d-c59a1305a71d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:07.306090 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-629f1069-899f-4fb3-9289-75184202b524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:07.306870 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-629f1069-899f-4fb3-9289-75184202b524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:07.325761 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-629f1069-899f-4fb3-9289-75184202b524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:07.326462 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-629f1069-899f-4fb3-9289-75184202b524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:07.327081 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-629f1069-899f-4fb3-9289-75184202b524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:07.332162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-629f1069-899f-4fb3-9289-75184202b524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:07.335070 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-629f1069-899f-4fb3-9289-75184202b524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:07.338374 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-629f1069-899f-4fb3-9289-75184202b524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:07.408101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4633c864-c35d-41ce-a81d-c59a1305a71d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:07.416697 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4633c864-c35d-41ce-a81d-c59a1305a71d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.245631 Oct 20 09:03:07.418889 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 218/406] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:07 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:07.501613 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:07.502650 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:07.512144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:07.512649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:07.513416 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:07.514276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:07.514917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:07.515343 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:07.561516 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-629f1069-899f-4fb3-9289-75184202b524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:07.572587 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-629f1069-899f-4fb3-9289-75184202b524 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1370 microversion: 2.1 time: 0.272654 Oct 20 09:03:07.573350 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 189/407] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:07 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1370 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:07.773834 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:07.784286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:07.785964 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:07.787456 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-67b82037-424d-439f-9f11-fcd5bb42d45f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1736 microversion: 2.20 time: 0.290186 Oct 20 09:03:07.803043 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 219/408] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:07 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1736 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:08.440505 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-65735052-58e8-42f8-8cec-cb2977f5d1bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:08.441914 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-65735052-58e8-42f8-8cec-cb2977f5d1bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:08.452593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65735052-58e8-42f8-8cec-cb2977f5d1bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:08.453290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65735052-58e8-42f8-8cec-cb2977f5d1bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:08.453868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65735052-58e8-42f8-8cec-cb2977f5d1bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:08.457121 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65735052-58e8-42f8-8cec-cb2977f5d1bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:08.457812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65735052-58e8-42f8-8cec-cb2977f5d1bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:08.458593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65735052-58e8-42f8-8cec-cb2977f5d1bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:08.599041 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9250eb39-f41e-4728-9e78-042b6eb2fcf7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:08.599660 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9250eb39-f41e-4728-9e78-042b6eb2fcf7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:08.613131 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9250eb39-f41e-4728-9e78-042b6eb2fcf7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:08.613634 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9250eb39-f41e-4728-9e78-042b6eb2fcf7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:08.614450 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9250eb39-f41e-4728-9e78-042b6eb2fcf7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:08.615383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9250eb39-f41e-4728-9e78-042b6eb2fcf7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:08.616126 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9250eb39-f41e-4728-9e78-042b6eb2fcf7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:08.618604 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9250eb39-f41e-4728-9e78-042b6eb2fcf7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:08.659855 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65735052-58e8-42f8-8cec-cb2977f5d1bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:08.668924 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-65735052-58e8-42f8-8cec-cb2977f5d1bf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.232219 Oct 20 09:03:08.682492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 190/409] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:08 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:08.698681 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-228f15fa-715e-4dd0-830c-4729f5c57535 req-7da34930-a9d2-4b79-98c1-dcca636845ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aae7b08a-4c9a-4d12-9598-193afab4ca00", "name": "network-vif-plugged", "status": "completed", "tag": "68cd5f68-1310-4334-b214-dc31475d3f8f"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:08.720807 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-228f15fa-715e-4dd0-830c-4729f5c57535 req-7da34930-a9d2-4b79-98c1-dcca636845ba service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:08.721460 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-228f15fa-715e-4dd0-830c-4729f5c57535 req-7da34930-a9d2-4b79-98c1-dcca636845ba service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:08.722663 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-228f15fa-715e-4dd0-830c-4729f5c57535 req-7da34930-a9d2-4b79-98c1-dcca636845ba service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:08.743469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-228f15fa-715e-4dd0-830c-4729f5c57535 req-7da34930-a9d2-4b79-98c1-dcca636845ba service nova] Creating event network-vif-plugged:68cd5f68-1310-4334-b214-dc31475d3f8f for instance aae7b08a-4c9a-4d12-9598-193afab4ca00 on hv-cinder-maste Oct 20 09:03:08.750065 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-228f15fa-715e-4dd0-830c-4729f5c57535 req-7da34930-a9d2-4b79-98c1-dcca636845ba service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.078729 Oct 20 09:03:08.750615 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 191/410] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:03:08.817294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:08.817988 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:08.830315 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:08.831407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:08.833362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:08.835124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:08.836463 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:08.837094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:08.858329 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9250eb39-f41e-4728-9e78-042b6eb2fcf7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:08.868784 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9250eb39-f41e-4728-9e78-042b6eb2fcf7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1370 microversion: 2.1 time: 0.273896 Oct 20 09:03:08.869364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 220/411] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:08 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1370 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:09.096995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:09.113639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:09.116046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:09.119002 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-98c69ee0-984d-4445-8c83-809c3559a47f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1736 microversion: 2.20 time: 0.305271 Oct 20 09:03:09.119956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 192/412] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:08 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1736 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:09.697777 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-85252903-ea93-4d57-a0f9-b0ab4c48ca1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:09.698392 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-85252903-ea93-4d57-a0f9-b0ab4c48ca1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:09.709370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85252903-ea93-4d57-a0f9-b0ab4c48ca1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:09.709370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85252903-ea93-4d57-a0f9-b0ab4c48ca1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:09.709370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85252903-ea93-4d57-a0f9-b0ab4c48ca1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:09.709370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85252903-ea93-4d57-a0f9-b0ab4c48ca1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:09.709695 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85252903-ea93-4d57-a0f9-b0ab4c48ca1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:09.709695 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85252903-ea93-4d57-a0f9-b0ab4c48ca1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:09.860191 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-85252903-ea93-4d57-a0f9-b0ab4c48ca1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:09.869616 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-85252903-ea93-4d57-a0f9-b0ab4c48ca1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.175803 Oct 20 09:03:09.870296 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 221/413] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:09 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:09.896833 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6157b277-df54-4f2f-877a-c9dd6064374b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:09.897454 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6157b277-df54-4f2f-877a-c9dd6064374b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:09.914425 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6157b277-df54-4f2f-877a-c9dd6064374b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:09.914969 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6157b277-df54-4f2f-877a-c9dd6064374b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:09.915412 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6157b277-df54-4f2f-877a-c9dd6064374b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:09.916367 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6157b277-df54-4f2f-877a-c9dd6064374b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:09.916821 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6157b277-df54-4f2f-877a-c9dd6064374b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:09.917253 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6157b277-df54-4f2f-877a-c9dd6064374b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:10.122121 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6157b277-df54-4f2f-877a-c9dd6064374b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:10.143268 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6157b277-df54-4f2f-877a-c9dd6064374b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1370 microversion: 2.1 time: 0.247467 Oct 20 09:03:10.160573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 193/414] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:09 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1370 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:10.163616 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:10.163616 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:10.176534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:10.177655 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:10.178512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:10.180545 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:10.182028 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:10.183194 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:10.345310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:10.356423 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:10.358570 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:10.360493 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1b7c20d5-a4ff-46b8-a4bf-374a7c11b00f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1736 microversion: 2.20 time: 0.206751 Oct 20 09:03:10.361213 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 222/415] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:10 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1736 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:10.777618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0fd1575d-b9bf-4fe8-b269-494fd434832b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:10.778304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0fd1575d-b9bf-4fe8-b269-494fd434832b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:10.781862 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-ed683184-effc-4a27-b47e-642e50b8f059 req-556907c5-f7fc-4263-b81a-8fad12a0029d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ea0966d3-1311-4690-a69f-d3df935a76e1", "name": "network-vif-plugged", "status": "completed", "tag": "b9bd9664-f1f8-415b-84b5-9c0f08f3fc00"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:10.784930 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0fd1575d-b9bf-4fe8-b269-494fd434832b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:10.785504 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0fd1575d-b9bf-4fe8-b269-494fd434832b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:10.786024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0fd1575d-b9bf-4fe8-b269-494fd434832b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:10.786832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0fd1575d-b9bf-4fe8-b269-494fd434832b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:10.787400 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0fd1575d-b9bf-4fe8-b269-494fd434832b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:10.788070 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0fd1575d-b9bf-4fe8-b269-494fd434832b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:10.793804 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-ed683184-effc-4a27-b47e-642e50b8f059 req-556907c5-f7fc-4263-b81a-8fad12a0029d service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:10.794288 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-ed683184-effc-4a27-b47e-642e50b8f059 req-556907c5-f7fc-4263-b81a-8fad12a0029d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:10.794892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-ed683184-effc-4a27-b47e-642e50b8f059 req-556907c5-f7fc-4263-b81a-8fad12a0029d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:10.807140 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-ed683184-effc-4a27-b47e-642e50b8f059 req-556907c5-f7fc-4263-b81a-8fad12a0029d service nova] Creating event network-vif-plugged:b9bd9664-f1f8-415b-84b5-9c0f08f3fc00 for instance ea0966d3-1311-4690-a69f-d3df935a76e1 on hv-cinder-maste Oct 20 09:03:10.815595 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-ed683184-effc-4a27-b47e-642e50b8f059 req-556907c5-f7fc-4263-b81a-8fad12a0029d service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043660 Oct 20 09:03:10.816880 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 194/416] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:03:10.900633 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ce57769f-07d4-4fb6-b6a9-e64b0f955865 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:10.901491 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ce57769f-07d4-4fb6-b6a9-e64b0f955865 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:10.922966 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce57769f-07d4-4fb6-b6a9-e64b0f955865 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:10.923466 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce57769f-07d4-4fb6-b6a9-e64b0f955865 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:10.923911 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce57769f-07d4-4fb6-b6a9-e64b0f955865 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:10.924621 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce57769f-07d4-4fb6-b6a9-e64b0f955865 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:10.925217 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce57769f-07d4-4fb6-b6a9-e64b0f955865 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:10.925649 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce57769f-07d4-4fb6-b6a9-e64b0f955865 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:10.991580 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-0fd1575d-b9bf-4fe8-b269-494fd434832b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lazy-loading 'services' on Instance uuid fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:03:11.113822 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ce57769f-07d4-4fb6-b6a9-e64b0f955865 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:11.122362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ce57769f-07d4-4fb6-b6a9-e64b0f955865 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1366 microversion: 2.1 time: 0.229334 Oct 20 09:03:11.125295 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 195/417] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:10 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1366 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:11.160075 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0fd1575d-b9bf-4fe8-b269-494fd434832b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "DELETE /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67/os-volume_attachments/f320b8eb-5558-4511-bf09-b963d2f19942" status: 202 len: 0 microversion: 2.1 time: 0.393235 Oct 20 09:03:11.160979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 223/418] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:03:10 2022] DELETE /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67/os-volume_attachments/f320b8eb-5558-4511-bf09-b963d2f19942 => generated 0 bytes in 395 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:03:11.200217 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2107dfc7-91ef-4163-a366-161d7c2b666e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:11.201111 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2107dfc7-91ef-4163-a366-161d7c2b666e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:11.211741 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2107dfc7-91ef-4163-a366-161d7c2b666e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:11.212239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2107dfc7-91ef-4163-a366-161d7c2b666e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:11.212678 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2107dfc7-91ef-4163-a366-161d7c2b666e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:11.213383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2107dfc7-91ef-4163-a366-161d7c2b666e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:11.213837 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2107dfc7-91ef-4163-a366-161d7c2b666e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:11.214263 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2107dfc7-91ef-4163-a366-161d7c2b666e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:11.393331 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:11.394071 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:11.406536 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2107dfc7-91ef-4163-a366-161d7c2b666e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:11.413156 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:11.413915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:11.414548 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:11.415441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:11.416064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:11.416388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2107dfc7-91ef-4163-a366-161d7c2b666e tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1370 microversion: 2.1 time: 0.220101 Oct 20 09:03:11.416933 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:11.422492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 196/419] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:11 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1370 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:11.615257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:11.628351 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:11.630713 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:11.632645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ea1600e6-1dc1-4398-ad9c-24bfabecdc77 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1736 microversion: 2.20 time: 0.247414 Oct 20 09:03:11.649943 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 224/420] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:11 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1736 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:12.150538 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-31aa392c-4e72-43ed-96ff-6fffd155cd08 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:12.151720 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-31aa392c-4e72-43ed-96ff-6fffd155cd08 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:12.158714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31aa392c-4e72-43ed-96ff-6fffd155cd08 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:12.159469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31aa392c-4e72-43ed-96ff-6fffd155cd08 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:12.159908 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31aa392c-4e72-43ed-96ff-6fffd155cd08 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:12.160636 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31aa392c-4e72-43ed-96ff-6fffd155cd08 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:12.161248 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31aa392c-4e72-43ed-96ff-6fffd155cd08 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:12.161842 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31aa392c-4e72-43ed-96ff-6fffd155cd08 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:12.313413 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-31aa392c-4e72-43ed-96ff-6fffd155cd08 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:12.324078 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-31aa392c-4e72-43ed-96ff-6fffd155cd08 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1556 microversion: 2.1 time: 0.177946 Oct 20 09:03:12.324670 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 197/421] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:12 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1556 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:12.348012 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3a329b0c-b2fc-4c49-9fcf-4247ddb4ff47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:12.349260 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3a329b0c-b2fc-4c49-9fcf-4247ddb4ff47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:12.355846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a329b0c-b2fc-4c49-9fcf-4247ddb4ff47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:12.356563 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a329b0c-b2fc-4c49-9fcf-4247ddb4ff47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:12.357239 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a329b0c-b2fc-4c49-9fcf-4247ddb4ff47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:12.357891 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a329b0c-b2fc-4c49-9fcf-4247ddb4ff47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:12.358339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a329b0c-b2fc-4c49-9fcf-4247ddb4ff47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:12.358925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a329b0c-b2fc-4c49-9fcf-4247ddb4ff47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:12.451498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5967a3e9-9546-459b-8a38-d979908d1311 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:12.451671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3a329b0c-b2fc-4c49-9fcf-4247ddb4ff47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.107731 Oct 20 09:03:12.452591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5967a3e9-9546-459b-8a38-d979908d1311 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:12.453163 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 225/422] 10.222.0.36 () {58 vars in 1275 bytes} [Thu Oct 20 09:03:12 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00/os-interface => generated 285 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:03:12.462430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5967a3e9-9546-459b-8a38-d979908d1311 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:12.463218 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5967a3e9-9546-459b-8a38-d979908d1311 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:12.463967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5967a3e9-9546-459b-8a38-d979908d1311 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:12.464928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5967a3e9-9546-459b-8a38-d979908d1311 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:12.466005 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5967a3e9-9546-459b-8a38-d979908d1311 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:12.466739 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5967a3e9-9546-459b-8a38-d979908d1311 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:12.645932 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5967a3e9-9546-459b-8a38-d979908d1311 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:12.655268 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5967a3e9-9546-459b-8a38-d979908d1311 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1370 microversion: 2.1 time: 0.207434 Oct 20 09:03:12.666367 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:12.667005 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:12.674001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 198/423] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:12 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1370 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:12.675620 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:12.676114 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:12.676552 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:12.677257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:12.677713 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:12.678135 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:12.976886 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:12.990528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:12.990528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:12.990528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-dbfd5daa-1afe-426c-aa49-cece47e7f554 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1736 microversion: 2.20 time: 0.327911 Oct 20 09:03:12.990966 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 226/424] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:12 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1736 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:13.686413 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0febd025-dfa8-478d-a6d0-ad4cb2474d52 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:13.687101 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0febd025-dfa8-478d-a6d0-ad4cb2474d52 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:13.702547 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0febd025-dfa8-478d-a6d0-ad4cb2474d52 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:13.708198 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-306272cf-ef91-4ebb-86ce-eabb98786b0e req-af554a9d-b5de-4311-9f5e-21205e5fee05 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aae7b08a-4c9a-4d12-9598-193afab4ca00", "tag": "68cd5f68-1310-4334-b214-dc31475d3f8f"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:13.709719 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0febd025-dfa8-478d-a6d0-ad4cb2474d52 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:13.715841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0febd025-dfa8-478d-a6d0-ad4cb2474d52 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:13.716565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0febd025-dfa8-478d-a6d0-ad4cb2474d52 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:13.717032 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0febd025-dfa8-478d-a6d0-ad4cb2474d52 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:13.717452 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0febd025-dfa8-478d-a6d0-ad4cb2474d52 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:13.722409 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-306272cf-ef91-4ebb-86ce-eabb98786b0e req-af554a9d-b5de-4311-9f5e-21205e5fee05 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:13.723292 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-306272cf-ef91-4ebb-86ce-eabb98786b0e req-af554a9d-b5de-4311-9f5e-21205e5fee05 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:13.724103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-306272cf-ef91-4ebb-86ce-eabb98786b0e req-af554a9d-b5de-4311-9f5e-21205e5fee05 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:13.740857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-306272cf-ef91-4ebb-86ce-eabb98786b0e req-af554a9d-b5de-4311-9f5e-21205e5fee05 service nova] Creating event network-changed:68cd5f68-1310-4334-b214-dc31475d3f8f for instance aae7b08a-4c9a-4d12-9598-193afab4ca00 on hv-cinder-maste Oct 20 09:03:13.746584 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-306272cf-ef91-4ebb-86ce-eabb98786b0e req-af554a9d-b5de-4311-9f5e-21205e5fee05 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044778 Oct 20 09:03:13.751447 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 227/425] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:03:13.943652 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0febd025-dfa8-478d-a6d0-ad4cb2474d52 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:13.962314 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0febd025-dfa8-478d-a6d0-ad4cb2474d52 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1370 microversion: 2.1 time: 0.284250 Oct 20 09:03:13.982711 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 199/426] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:13 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1370 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:14.018644 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:14.019350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:14.029191 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:14.029690 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:14.030131 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:14.030879 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:14.031334 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:14.031762 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:14.202256 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:14.215458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:14.217962 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:14.219691 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4c1997b3-921b-49f3-9e99-c10a019852d7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1736 microversion: 2.20 time: 0.206208 Oct 20 09:03:14.231512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 228/427] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:14 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1736 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:14.500174 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d24c7aa2-2969-4210-8b18-29a0ca5cd31e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:14.500947 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d24c7aa2-2969-4210-8b18-29a0ca5cd31e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:14.510296 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d24c7aa2-2969-4210-8b18-29a0ca5cd31e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:14.511093 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d24c7aa2-2969-4210-8b18-29a0ca5cd31e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:14.511704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d24c7aa2-2969-4210-8b18-29a0ca5cd31e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:14.512562 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d24c7aa2-2969-4210-8b18-29a0ca5cd31e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:14.513188 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d24c7aa2-2969-4210-8b18-29a0ca5cd31e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:14.513771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d24c7aa2-2969-4210-8b18-29a0ca5cd31e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:14.542827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d24c7aa2-2969-4210-8b18-29a0ca5cd31e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:03:14.607458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:14.657692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d24c7aa2-2969-4210-8b18-29a0ca5cd31e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "DELETE /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 204 len: 0 microversion: 2.1 time: 0.161057 Oct 20 09:03:14.667253 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-65d2d242-9151-4e04-9561-9fd7ea84615e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:14.667915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-65d2d242-9151-4e04-9561-9fd7ea84615e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:14.675565 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-65d2d242-9151-4e04-9561-9fd7ea84615e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:14.677139 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 200/428] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:03:14 2022] DELETE /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 20 09:03:14.680133 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-65d2d242-9151-4e04-9561-9fd7ea84615e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:14.681466 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-65d2d242-9151-4e04-9561-9fd7ea84615e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:14.684269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-65d2d242-9151-4e04-9561-9fd7ea84615e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:14.685642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-65d2d242-9151-4e04-9561-9fd7ea84615e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:14.687701 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-65d2d242-9151-4e04-9561-9fd7ea84615e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:14.864592 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-65d2d242-9151-4e04-9561-9fd7ea84615e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:14.879944 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-65d2d242-9151-4e04-9561-9fd7ea84615e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1559 microversion: 2.1 time: 0.217270 Oct 20 09:03:14.880537 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 229/429] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:14 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1559 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:14.997960 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c729c8a4-d969-4771-9fb2-cab9906f2f45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:14.998698 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c729c8a4-d969-4771-9fb2-cab9906f2f45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:15.008573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c729c8a4-d969-4771-9fb2-cab9906f2f45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:15.009068 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c729c8a4-d969-4771-9fb2-cab9906f2f45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:15.009723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c729c8a4-d969-4771-9fb2-cab9906f2f45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:15.011024 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c729c8a4-d969-4771-9fb2-cab9906f2f45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:15.011642 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c729c8a4-d969-4771-9fb2-cab9906f2f45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:15.013030 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c729c8a4-d969-4771-9fb2-cab9906f2f45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:15.226174 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c729c8a4-d969-4771-9fb2-cab9906f2f45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:15.239337 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c729c8a4-d969-4771-9fb2-cab9906f2f45 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1563 microversion: 2.1 time: 0.245864 Oct 20 09:03:15.240730 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 201/430] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:14 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1563 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:15.257917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:15.258826 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:15.270883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-18d346b3-3cda-4852-a104-680cd8bbe9d7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:15.271233 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-18d346b3-3cda-4852-a104-680cd8bbe9d7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:15.276218 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:15.277178 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:15.277736 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:15.278700 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18d346b3-3cda-4852-a104-680cd8bbe9d7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:15.279162 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:15.279520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18d346b3-3cda-4852-a104-680cd8bbe9d7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:15.280105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18d346b3-3cda-4852-a104-680cd8bbe9d7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:15.280425 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:15.281134 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18d346b3-3cda-4852-a104-680cd8bbe9d7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:15.281456 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:15.281891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18d346b3-3cda-4852-a104-680cd8bbe9d7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:15.282460 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18d346b3-3cda-4852-a104-680cd8bbe9d7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:15.392856 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-18d346b3-3cda-4852-a104-680cd8bbe9d7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.127517 Oct 20 09:03:15.394174 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 202/431] 10.222.0.36 () {58 vars in 1275 bytes} [Thu Oct 20 09:03:15 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-interface => generated 285 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:03:15.500967 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:15.523046 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:15.525173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:15.527541 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-971196ff-5ccf-4ff9-be86-cbe171eb0445 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1736 microversion: 2.20 time: 0.273113 Oct 20 09:03:15.528113 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 230/432] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:15 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1736 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:15.912832 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4459ee79-b7e4-45ea-8492-837c3531b835 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:15.914211 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4459ee79-b7e4-45ea-8492-837c3531b835 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:15.920747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4459ee79-b7e4-45ea-8492-837c3531b835 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:15.921512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4459ee79-b7e4-45ea-8492-837c3531b835 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:15.924009 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4459ee79-b7e4-45ea-8492-837c3531b835 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:15.925394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4459ee79-b7e4-45ea-8492-837c3531b835 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:15.926214 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4459ee79-b7e4-45ea-8492-837c3531b835 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:15.926880 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4459ee79-b7e4-45ea-8492-837c3531b835 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:16.485639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4459ee79-b7e4-45ea-8492-837c3531b835 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:16.495765 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4459ee79-b7e4-45ea-8492-837c3531b835 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1513 microversion: 2.1 time: 0.591318 Oct 20 09:03:16.510871 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 203/433] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:15 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1513 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:16.557716 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:16.559475 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:16.574833 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:16.583112 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:16.592622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:16.593924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:16.594531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:16.595070 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:16.812736 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:16.824736 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:16.828196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:16.829764 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ab9769bc-af1c-4a23-975e-928d74d2db51 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1736 microversion: 2.20 time: 0.276583 Oct 20 09:03:16.836491 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 231/434] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:16 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1736 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:17.036411 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-c1db2009-8bf1-443d-a98a-c3448cd83ca7 req-bad8f030-1bd0-4c29-8c88-9376e2352e62 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ea0966d3-1311-4690-a69f-d3df935a76e1", "tag": "b9bd9664-f1f8-415b-84b5-9c0f08f3fc00"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:17.042193 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-c1db2009-8bf1-443d-a98a-c3448cd83ca7 req-bad8f030-1bd0-4c29-8c88-9376e2352e62 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:17.042877 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-c1db2009-8bf1-443d-a98a-c3448cd83ca7 req-bad8f030-1bd0-4c29-8c88-9376e2352e62 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:17.043486 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-c1db2009-8bf1-443d-a98a-c3448cd83ca7 req-bad8f030-1bd0-4c29-8c88-9376e2352e62 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:17.059176 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-c1db2009-8bf1-443d-a98a-c3448cd83ca7 req-bad8f030-1bd0-4c29-8c88-9376e2352e62 service nova] Creating event network-changed:b9bd9664-f1f8-415b-84b5-9c0f08f3fc00 for instance ea0966d3-1311-4690-a69f-d3df935a76e1 on hv-cinder-maste Oct 20 09:03:17.064343 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-c1db2009-8bf1-443d-a98a-c3448cd83ca7 req-bad8f030-1bd0-4c29-8c88-9376e2352e62 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037572 Oct 20 09:03:17.065314 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 204/435] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:03:17.536234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-499e7045-dca9-4672-b828-e90c80a8c0dc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:17.537138 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-499e7045-dca9-4672-b828-e90c80a8c0dc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:17.548404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-499e7045-dca9-4672-b828-e90c80a8c0dc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:17.548934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-499e7045-dca9-4672-b828-e90c80a8c0dc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:17.549377 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-499e7045-dca9-4672-b828-e90c80a8c0dc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:17.550090 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-499e7045-dca9-4672-b828-e90c80a8c0dc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:17.550591 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-499e7045-dca9-4672-b828-e90c80a8c0dc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:17.550999 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-499e7045-dca9-4672-b828-e90c80a8c0dc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:17.792989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-499e7045-dca9-4672-b828-e90c80a8c0dc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:17.806855 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-499e7045-dca9-4672-b828-e90c80a8c0dc tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1513 microversion: 2.1 time: 0.283518 Oct 20 09:03:17.823625 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 232/436] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:17 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1513 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:17.873744 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:17.877740 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:17.902584 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:17.910286 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:17.911029 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:17.911978 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:17.916867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:17.917532 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:18.300097 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:18.315297 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:18.317074 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:18.318267 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-45b47d2f-c38a-4c27-9ff0-5026945a16ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1736 microversion: 2.20 time: 0.449320 Oct 20 09:03:18.318872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 205/437] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:17 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1736 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:18.836682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2518eecb-8af3-4715-9609-2d746db205b2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:18.840051 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2518eecb-8af3-4715-9609-2d746db205b2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:18.842819 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:18.850960 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2518eecb-8af3-4715-9609-2d746db205b2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:18.851942 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2518eecb-8af3-4715-9609-2d746db205b2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:18.852872 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2518eecb-8af3-4715-9609-2d746db205b2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:18.853864 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2518eecb-8af3-4715-9609-2d746db205b2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:18.854376 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2518eecb-8af3-4715-9609-2d746db205b2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:18.856358 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2518eecb-8af3-4715-9609-2d746db205b2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:18.937009 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2518eecb-8af3-4715-9609-2d746db205b2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:18.945701 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2518eecb-8af3-4715-9609-2d746db205b2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 200 len: 1304 microversion: 2.1 time: 0.117033 Oct 20 09:03:18.946284 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 233/438] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:18 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 1304 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:19.080310 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-d7d1c3c0-e633-4565-9097-c8fd5beb5f52 req-85298358-4cdb-4282-8381-4807d059d03d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fe511cdb-e20c-4ec9-a37b-1eb239d3ed67", "name": "network-vif-deleted", "tag": "6bd83ee3-a5b5-413b-9b18-8034518260a1"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:19.090971 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-d7d1c3c0-e633-4565-9097-c8fd5beb5f52 req-85298358-4cdb-4282-8381-4807d059d03d service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:19.091452 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-d7d1c3c0-e633-4565-9097-c8fd5beb5f52 req-85298358-4cdb-4282-8381-4807d059d03d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:19.091883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-d7d1c3c0-e633-4565-9097-c8fd5beb5f52 req-85298358-4cdb-4282-8381-4807d059d03d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:19.103687 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-d7d1c3c0-e633-4565-9097-c8fd5beb5f52 req-85298358-4cdb-4282-8381-4807d059d03d service nova] Creating event network-vif-deleted:6bd83ee3-a5b5-413b-9b18-8034518260a1 for instance fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 on hv-cinder-maste Oct 20 09:03:19.108551 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-d7d1c3c0-e633-4565-9097-c8fd5beb5f52 req-85298358-4cdb-4282-8381-4807d059d03d service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031868 Oct 20 09:03:19.109323 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 206/439] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:03:19.344374 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:19.344977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:19.351593 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:19.352072 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:19.352513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:19.353229 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:19.353674 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:19.354093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:19.358378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:19.529346 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:19.537391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:19.539395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:19.540573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9df01526-8e45-49a0-a1d7-79dc3f78500c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1718 microversion: 2.20 time: 0.200299 Oct 20 09:03:19.541128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 234/440] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:19 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1718 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:19.970135 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:19.971128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: fe511cdb-e20c-4ec9-a37b-1eb239d3ed67] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:19.981370 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:19.982147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:19.982642 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:19.983529 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:19.984171 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:19.984605 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:20.002232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] HTTP exception thrown: Instance fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 could not be found. Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Request method failure captured: Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 HTTP/1.1 Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ6AAjqBCauEsn7-wVxFXyE5gkDP7ByGV9OJJcihX6NnQdtOCcweoduHZl0pTDp5mRnTkb3HVtP5FXtFisjEzcaT2-weJ4wxtTpLpMzepcqszoX2CJAOGbaor1x3yy44yWllx4hJ4EheGd-OtFLJGzyPVOSXc55J6lPkNDY43Dhs2cg Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 9dc1a3b80bd34c8e8f691a9b47457f35 Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1293092775 Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:03:20.004407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/9dc1a3b80bd34c8e8f691a9b47457f35"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_9dc1a3b80bd34c8e8f691a9b47457f35", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/9dc1a3b80bd34c8e8f691a9b47457f35"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/9dc1a3b80bd34c8e8f691a9b47457f35"}]}] Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-VolumesSnapshotTestJSON-1293092775 Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 9dc1a3b80bd34c8e8f691a9b47457f35 Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1293092775 Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-VolumesSnapshotTestJSON-1293092775-project Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 7e0951446ad449d6b725fe6cc14df1f2 Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-VolumesSnapshotTestJSON-1293092775-project Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 could not be found. Oct 20 09:03:20.005990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': 'fe511cdb-e20c-4ec9-a37b-1eb239d3ed67'} Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 could not be found. Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:03:20.007001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 could not be found. Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 could not be found. Oct 20 09:03:20.008289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:03:20.010839 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Returning 404 to user: Instance fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:03:20.012009 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7802640d-e688-44aa-b41f-828f0644a80c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67" status: 404 len: 111 microversion: 2.1 time: 0.047257 Oct 20 09:03:20.012866 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 207/441] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:19 2022] GET /compute/v2.1/servers/fe511cdb-e20c-4ec9-a37b-1eb239d3ed67 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:03:20.566154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:20.566784 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:20.573661 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:20.574140 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:20.574600 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:20.575289 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:20.575737 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:20.576153 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:20.593838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-6076c2a9-2eb8-4a55-ad13-6b7df73ca672 req-4b288baf-b111-4537-a564-d8c9fe8b5544 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f8d61907-24ca-4e74-917e-d218d0f89557", "name": "network-vif-unplugged", "status": "completed", "tag": "6ba2009e-08be-4c65-a1d5-5e9548c8d407"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:20.605748 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6076c2a9-2eb8-4a55-ad13-6b7df73ca672 req-4b288baf-b111-4537-a564-d8c9fe8b5544 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:20.606229 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6076c2a9-2eb8-4a55-ad13-6b7df73ca672 req-4b288baf-b111-4537-a564-d8c9fe8b5544 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:20.606684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6076c2a9-2eb8-4a55-ad13-6b7df73ca672 req-4b288baf-b111-4537-a564-d8c9fe8b5544 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:20.618851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-6076c2a9-2eb8-4a55-ad13-6b7df73ca672 req-4b288baf-b111-4537-a564-d8c9fe8b5544 service nova] Creating event network-vif-unplugged:6ba2009e-08be-4c65-a1d5-5e9548c8d407 for instance f8d61907-24ca-4e74-917e-d218d0f89557 on hv-cinder-maste Oct 20 09:03:20.623887 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-6076c2a9-2eb8-4a55-ad13-6b7df73ca672 req-4b288baf-b111-4537-a564-d8c9fe8b5544 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.033949 Oct 20 09:03:20.624533 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 208/442] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:20 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:03:20.710914 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:20.719292 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:20.720739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:20.721885 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-be0a62f3-f662-425c-85d2-e55bfeb33997 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1718 microversion: 2.20 time: 0.159605 Oct 20 09:03:20.722437 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 235/443] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:20 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1718 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:21.747759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:21.748521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:21.754735 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:21.755210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:21.755638 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:21.756317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:21.757407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:21.757777 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:21.891421 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:21.900226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:21.901676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:21.902955 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-56c9ca1d-17a3-45f5-aa40-cedc1fda7086 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1718 microversion: 2.20 time: 0.159891 Oct 20 09:03:21.903570 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 209/444] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:21 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1718 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:22.927290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:22.927994 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:22.934679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:22.935220 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:22.935653 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:22.936329 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:22.936773 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:22.937188 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:23.082803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:23.092727 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:23.094286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:23.095554 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-71612821-c65d-467c-9cae-15953703037b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1718 microversion: 2.20 time: 0.172118 Oct 20 09:03:23.097004 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 236/445] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:22 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1718 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:24.121503 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:24.122107 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:24.128738 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:24.129224 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:24.129658 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:24.130462 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:24.131206 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:24.132208 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:24.342548 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:24.351903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:24.353400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:24.354656 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-10d2e370-e2f9-45bf-8b57-70450d7191cb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1721 microversion: 2.20 time: 0.237008 Oct 20 09:03:24.355545 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 210/446] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:24 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1721 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:24.390850 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4b6a4ac8-67ba-44a7-9b07-1a784526a019 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "796139d0-582f-43cf-a128-27d67f6063fb"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:24.395421 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4b6a4ac8-67ba-44a7-9b07-1a784526a019 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:24.418912 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4b6a4ac8-67ba-44a7-9b07-1a784526a019 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:24.419404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4b6a4ac8-67ba-44a7-9b07-1a784526a019 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:24.419838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4b6a4ac8-67ba-44a7-9b07-1a784526a019 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:24.420539 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4b6a4ac8-67ba-44a7-9b07-1a784526a019 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:24.420986 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4b6a4ac8-67ba-44a7-9b07-1a784526a019 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:24.421434 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4b6a4ac8-67ba-44a7-9b07-1a784526a019 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:24.773870 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4b6a4ac8-67ba-44a7-9b07-1a784526a019 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "POST /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.390633 Oct 20 09:03:24.780747 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 237/447] 10.222.0.36 () {62 vars in 1364 bytes} [Thu Oct 20 09:03:24 2022] POST /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments => generated 188 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 09:03:24.941158 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0a4be0a4-6bbf-4a64-a40a-0889abc36fe9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:24.941757 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0a4be0a4-6bbf-4a64-a40a-0889abc36fe9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:24.948102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a4be0a4-6bbf-4a64-a40a-0889abc36fe9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:24.948590 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a4be0a4-6bbf-4a64-a40a-0889abc36fe9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:24.949218 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a4be0a4-6bbf-4a64-a40a-0889abc36fe9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:24.951031 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a4be0a4-6bbf-4a64-a40a-0889abc36fe9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:24.951492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a4be0a4-6bbf-4a64-a40a-0889abc36fe9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:24.952475 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a4be0a4-6bbf-4a64-a40a-0889abc36fe9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:25.005776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0a4be0a4-6bbf-4a64-a40a-0889abc36fe9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=119068) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 20 09:03:25.111675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0a4be0a4-6bbf-4a64-a40a-0889abc36fe9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "POST /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/action" status: 202 len: 0 microversion: 2.20 time: 0.174366 Oct 20 09:03:25.114223 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 211/448] 10.222.0.36 () {62 vars in 1319 bytes} [Thu Oct 20 09:03:24 2022] POST /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/action => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:03:25.123055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:25.123691 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:25.133113 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:25.134205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:25.135292 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:25.136015 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:25.137264 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:25.138012 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:25.385412 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:25.403656 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:25.405849 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:25.407874 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-82d8983d-b5b6-4aa7-9718-ec5735b2c683 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1747 microversion: 2.20 time: 0.289323 Oct 20 09:03:25.424667 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 238/449] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:25 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1747 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:26.437374 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:26.438985 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:26.447523 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:26.448273 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:26.449100 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:26.450508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:26.451184 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:26.451802 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:26.595108 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:26.610655 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:26.612469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:26.614090 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-011fec12-2b44-490b-86fb-6b68f7bc1638 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1747 microversion: 2.20 time: 0.181069 Oct 20 09:03:26.615317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 212/450] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:26 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1747 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:27.646924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:27.648913 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:27.657128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:27.659652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:27.660474 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:27.661831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:27.662765 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:27.666620 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:27.851894 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:27.861447 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:27.863218 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:27.864736 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e5af8794-64aa-4218-8573-245a88732b33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1747 microversion: 2.20 time: 0.222508 Oct 20 09:03:27.865516 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 239/451] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:27 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1747 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:28.896801 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:28.898431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:28.905279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:28.905796 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:28.906230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:28.913776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:28.914557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:28.915767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:29.130912 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:29.146317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:29.148484 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:29.149674 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ddffdfcf-1c24-4756-bd27-44f2dc143797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1747 microversion: 2.20 time: 0.260822 Oct 20 09:03:29.150236 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 213/452] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:28 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1747 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:30.177509 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:30.180450 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:30.196947 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:30.197442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:30.197879 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:30.198611 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:30.199058 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:30.199472 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:30.344782 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1419431008", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "networks": [{"uuid": "7fdc4269-1e8c-4b8f-884b-33c74a51bd63"}]}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:30.450823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] validate_networks() for [('7fdc4269-1e8c-4b8f-884b-33c74a51bd63', None, None, None, None, None)] {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:03:30.580079 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:30.598589 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:30.603618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:30.605403 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6df63585-fe6c-4d38-807c-314f995d843d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1749 microversion: 2.20 time: 0.432094 Oct 20 09:03:30.606517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 240/453] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:30 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1749 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:31.029500 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:03:31.029937 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:03:31.032686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:03:31.033377 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:03:31.033805 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:03:31.047205 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:31.047683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:31.048179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:31.048937 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:31.049624 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:31.050051 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:31.076439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Getting quotas for project 9dc1a3b80bd34c8e8f691a9b47457f35. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:03:31.081330 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Getting quotas for user 7e0951446ad449d6b725fe6cc14df1f2 and project 9dc1a3b80bd34c8e8f691a9b47457f35. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:03:31.095151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:03:31.098571 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:03:31.187797 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f1e0ecae-5e95-407c-b38d-9ea94d33c409 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.848393 Oct 20 09:03:31.190755 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 214/454] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:03:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 852 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:03:31.206157 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-26e0fd1c-4e2c-48b2-9ed6-c98b85a9b567 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:31.206801 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-26e0fd1c-4e2c-48b2-9ed6-c98b85a9b567 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:31.321019 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-26e0fd1c-4e2c-48b2-9ed6-c98b85a9b567 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:31.329617 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-26e0fd1c-4e2c-48b2-9ed6-c98b85a9b567 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1219 microversion: 2.1 time: 0.130480 Oct 20 09:03:31.332540 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 241/455] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:31 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1219 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:31.632555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:31.633731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:31.644136 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:31.644629 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:31.645071 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:31.645777 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:31.651492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:31.651928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:31.698851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-409dcede-3443-492a-a805-4fe4b624d094 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:31.699663 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-409dcede-3443-492a-a805-4fe4b624d094 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:31.716073 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-409dcede-3443-492a-a805-4fe4b624d094 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:31.724361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-409dcede-3443-492a-a805-4fe4b624d094 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:31.725773 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-409dcede-3443-492a-a805-4fe4b624d094 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:31.726656 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-409dcede-3443-492a-a805-4fe4b624d094 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:31.729610 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-409dcede-3443-492a-a805-4fe4b624d094 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:31.730155 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-409dcede-3443-492a-a805-4fe4b624d094 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:31.785549 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-409dcede-3443-492a-a805-4fe4b624d094 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/ips" status: 200 len: 146 microversion: 2.1 time: 0.087946 Oct 20 09:03:31.788111 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 242/456] 10.222.0.36 () {58 vars in 1248 bytes} [Thu Oct 20 09:03:31 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/ips => generated 146 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:03:31.861772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:31.873400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:31.875676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:31.877528 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d83b2d04-5573-4df4-a14c-51834e1a431b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.249661 Oct 20 09:03:31.878254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 215/457] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:31 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:32.360182 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ea8eba6c-38cb-4d6d-819e-6b2a0c2f54f1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:32.360981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ea8eba6c-38cb-4d6d-819e-6b2a0c2f54f1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:32.377619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea8eba6c-38cb-4d6d-819e-6b2a0c2f54f1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:32.379814 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea8eba6c-38cb-4d6d-819e-6b2a0c2f54f1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:32.380705 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea8eba6c-38cb-4d6d-819e-6b2a0c2f54f1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:32.383205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea8eba6c-38cb-4d6d-819e-6b2a0c2f54f1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:32.383993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea8eba6c-38cb-4d6d-819e-6b2a0c2f54f1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:32.390350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea8eba6c-38cb-4d6d-819e-6b2a0c2f54f1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:32.846949 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ea8eba6c-38cb-4d6d-819e-6b2a0c2f54f1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:32.861484 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ea8eba6c-38cb-4d6d-819e-6b2a0c2f54f1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1271 microversion: 2.1 time: 0.507804 Oct 20 09:03:32.862072 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 243/458] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:32 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1271 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:32.910521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:32.911843 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:32.914836 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=88,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:32.920171 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:32.920660 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:32.921089 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:32.922019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:32.922460 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:32.923048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:33.091114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:33.101155 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:33.102816 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:33.104181 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-abd73ef9-a919-4a40-ae23-b33508716ce2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.197578 Oct 20 09:03:33.110491 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 216/459] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:32 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:33.475151 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-99d697de-9b46-4f62-9ee3-640359f1f0b3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Action: 'create', calling method: .version_select at 0x7f5afbdc4820>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1088348766", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO8PUvCm7sxm2REft53ky0ZYrEarOqK3+RhcD6cdiWCMtGDrNicZlProno5VaIWovHfEhpisOoqNfiFX7EQF7qho9ZBACpXHHZPe0jv8PSqmu1Lv5oVWSCiOlaQxCfM/Ng=="}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:33.495105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-99d697de-9b46-4f62-9ee3-640359f1f0b3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Getting quotas for project 98827dbef155480b9f26792b70014b4b. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:03:33.500155 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-99d697de-9b46-4f62-9ee3-640359f1f0b3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Getting quotas for user 65ca2eae6db24bb094afaacdfb5c1980 and project 98827dbef155480b9f26792b70014b4b. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:03:33.524753 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-99d697de-9b46-4f62-9ee3-640359f1f0b3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.276185 Oct 20 09:03:33.525587 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 244/460] 10.222.0.36 () {60 vars in 1159 bytes} [Thu Oct 20 09:03:33 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:03:33.887916 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-aa81907e-6889-40c1-9cee-1f27c487d621 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:33.889050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-aa81907e-6889-40c1-9cee-1f27c487d621 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:33.896178 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa81907e-6889-40c1-9cee-1f27c487d621 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:33.897434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa81907e-6889-40c1-9cee-1f27c487d621 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:33.898069 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa81907e-6889-40c1-9cee-1f27c487d621 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:33.899240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa81907e-6889-40c1-9cee-1f27c487d621 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:33.899944 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa81907e-6889-40c1-9cee-1f27c487d621 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:33.900579 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa81907e-6889-40c1-9cee-1f27c487d621 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:34.002422 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-aa81907e-6889-40c1-9cee-1f27c487d621 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:34.021602 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-aa81907e-6889-40c1-9cee-1f27c487d621 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1277 microversion: 2.1 time: 0.140749 Oct 20 09:03:34.027919 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 217/461] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:33 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1277 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:34.140637 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:34.141526 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:34.157806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:34.160162 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:34.161274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:34.164309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:34.165845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:34.166537 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:34.643698 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:34.664160 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:34.670263 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:34.671758 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-07fdeeb6-39a3-4f75-9b1b-f8bf56d72d6e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.541426 Oct 20 09:03:34.694597 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 245/462] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:34 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:35.050952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6250e5be-0cbc-43cc-9a33-fb84dba9717d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:35.052189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6250e5be-0cbc-43cc-9a33-fb84dba9717d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:35.064800 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6250e5be-0cbc-43cc-9a33-fb84dba9717d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:35.067573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6250e5be-0cbc-43cc-9a33-fb84dba9717d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:35.071954 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6250e5be-0cbc-43cc-9a33-fb84dba9717d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:35.074229 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6250e5be-0cbc-43cc-9a33-fb84dba9717d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:35.076278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6250e5be-0cbc-43cc-9a33-fb84dba9717d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:35.076905 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6250e5be-0cbc-43cc-9a33-fb84dba9717d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:35.355427 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6250e5be-0cbc-43cc-9a33-fb84dba9717d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:35.370154 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6250e5be-0cbc-43cc-9a33-fb84dba9717d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.323652 Oct 20 09:03:35.374242 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 218/463] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:35 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:35.720275 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:35.720886 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:35.733372 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:35.733866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:35.734305 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:35.738803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:35.739282 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:35.739971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:36.247360 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:36.295628 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:36.300240 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:36.311547 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b6f38731-0004-4426-a2af-0175e892e6a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.605465 Oct 20 09:03:36.313249 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 246/464] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:35 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 611 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:36.399287 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-db84458c-a9cb-49fe-9259-eaa9657cc1ea tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:36.399899 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-db84458c-a9cb-49fe-9259-eaa9657cc1ea tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:36.415158 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-db84458c-a9cb-49fe-9259-eaa9657cc1ea tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:36.420099 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-db84458c-a9cb-49fe-9259-eaa9657cc1ea tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:36.420696 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-db84458c-a9cb-49fe-9259-eaa9657cc1ea tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:36.421674 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-db84458c-a9cb-49fe-9259-eaa9657cc1ea tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:36.422267 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-db84458c-a9cb-49fe-9259-eaa9657cc1ea tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:36.422879 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-db84458c-a9cb-49fe-9259-eaa9657cc1ea tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:36.463620 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-f659d63e-9c71-4e5b-85eb-7f7ff3f8797f req-af8368eb-e519-45e8-8d73-f603267a2121 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "98592479-f7da-4df7-ad14-c6aea41d81e0", "tag": "eeeae5ec-ff15-4944-a558-e9f736d0ca1e"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:36.488100 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-f659d63e-9c71-4e5b-85eb-7f7ff3f8797f req-af8368eb-e519-45e8-8d73-f603267a2121 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:36.488839 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-f659d63e-9c71-4e5b-85eb-7f7ff3f8797f req-af8368eb-e519-45e8-8d73-f603267a2121 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:36.489424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-f659d63e-9c71-4e5b-85eb-7f7ff3f8797f req-af8368eb-e519-45e8-8d73-f603267a2121 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:36.532882 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-f659d63e-9c71-4e5b-85eb-7f7ff3f8797f req-af8368eb-e519-45e8-8d73-f603267a2121 service nova] Creating event network-changed:eeeae5ec-ff15-4944-a558-e9f736d0ca1e for instance 98592479-f7da-4df7-ad14-c6aea41d81e0 on hv-cinder-maste Oct 20 09:03:36.543533 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-f659d63e-9c71-4e5b-85eb-7f7ff3f8797f req-af8368eb-e519-45e8-8d73-f603267a2121 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084770 Oct 20 09:03:36.544790 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 247/465] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:03:36.829203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-db84458c-a9cb-49fe-9259-eaa9657cc1ea tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:36.832049 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:36.839917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-db84458c-a9cb-49fe-9259-eaa9657cc1ea tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.446489 Oct 20 09:03:36.859203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 219/466] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:36 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:37.346849 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:37.347626 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:37.355646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:37.356135 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:37.356566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:37.357265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:37.357711 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:37.358131 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:37.662937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:37.673032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:37.675017 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:37.676851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-40409ff0-0ee2-43da-9f7d-c3eddf13431e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.334182 Oct 20 09:03:37.679831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 248/467] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:37 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:37.876131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ec27ab4c-84b9-4654-94a9-bfa6bdc4521e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:37.877725 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ec27ab4c-84b9-4654-94a9-bfa6bdc4521e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:37.889236 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ec27ab4c-84b9-4654-94a9-bfa6bdc4521e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:37.890134 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ec27ab4c-84b9-4654-94a9-bfa6bdc4521e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:37.890812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ec27ab4c-84b9-4654-94a9-bfa6bdc4521e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:37.891880 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ec27ab4c-84b9-4654-94a9-bfa6bdc4521e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:37.892571 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ec27ab4c-84b9-4654-94a9-bfa6bdc4521e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:37.893182 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ec27ab4c-84b9-4654-94a9-bfa6bdc4521e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:37.976272 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-502b6248-7e6a-4ef4-9b69-f7b954d22c59 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bcaf282a-ef82-4a51-85a4-da738df00b21"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:37.980056 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-502b6248-7e6a-4ef4-9b69-f7b954d22c59 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:38.009283 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-502b6248-7e6a-4ef4-9b69-f7b954d22c59 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:38.009785 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-502b6248-7e6a-4ef4-9b69-f7b954d22c59 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:38.010225 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-502b6248-7e6a-4ef4-9b69-f7b954d22c59 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:38.010966 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-502b6248-7e6a-4ef4-9b69-f7b954d22c59 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:38.011418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-502b6248-7e6a-4ef4-9b69-f7b954d22c59 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:38.011843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-502b6248-7e6a-4ef4-9b69-f7b954d22c59 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:38.138329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ec27ab4c-84b9-4654-94a9-bfa6bdc4521e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:38.146494 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ec27ab4c-84b9-4654-94a9-bfa6bdc4521e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.277452 Oct 20 09:03:38.147708 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 220/468] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:37 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:38.707492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:38.708100 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:38.714196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:38.714732 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:38.715498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:38.716294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:38.716772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:38.717204 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:38.871322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:38.882064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:38.884550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:38.885948 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2b471081-c370-4601-ae46-e6d1617f5276 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.182434 Oct 20 09:03:38.886530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 221/469] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:38 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:39.169870 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-357ded98-642e-406b-82a8-4802fe4027e4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:39.170403 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-357ded98-642e-406b-82a8-4802fe4027e4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:39.177452 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-357ded98-642e-406b-82a8-4802fe4027e4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:39.178450 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-357ded98-642e-406b-82a8-4802fe4027e4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:39.178914 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-357ded98-642e-406b-82a8-4802fe4027e4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:39.179607 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-357ded98-642e-406b-82a8-4802fe4027e4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:39.180054 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-357ded98-642e-406b-82a8-4802fe4027e4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:39.180479 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-357ded98-642e-406b-82a8-4802fe4027e4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:39.317093 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-357ded98-642e-406b-82a8-4802fe4027e4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:39.325894 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-357ded98-642e-406b-82a8-4802fe4027e4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.160189 Oct 20 09:03:39.326583 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 222/470] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:39 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:39.911354 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:39.911983 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:39.919392 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:39.920417 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:39.921381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:39.922572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:39.923319 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:39.923968 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:40.078683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:40.087447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:40.089227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:40.090700 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4f27ab0c-4efb-4ffc-8469-ece262b13a07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.183002 Oct 20 09:03:40.091259 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 223/471] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:39 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:40.349527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d68715e7-8af3-4475-8aee-591221963c61 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:40.350053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d68715e7-8af3-4475-8aee-591221963c61 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:40.356639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d68715e7-8af3-4475-8aee-591221963c61 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:40.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d68715e7-8af3-4475-8aee-591221963c61 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:40.357561 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d68715e7-8af3-4475-8aee-591221963c61 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:40.358259 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d68715e7-8af3-4475-8aee-591221963c61 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:40.358735 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d68715e7-8af3-4475-8aee-591221963c61 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:40.359159 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d68715e7-8af3-4475-8aee-591221963c61 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:40.503377 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d68715e7-8af3-4475-8aee-591221963c61 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:40.521334 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d68715e7-8af3-4475-8aee-591221963c61 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.173402 Oct 20 09:03:40.526129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 224/472] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:40 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:41.119560 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:41.124525 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:41.126458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:41.127155 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=6 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:41.133989 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:41.134498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:41.134955 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:41.135684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:41.136133 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:41.136551 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:41.274999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:41.284544 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:41.286139 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:41.287644 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f950bd56-2ee1-4e61-b065-9707092da032 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.176227 Oct 20 09:03:41.288530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 225/473] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:41 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:41.545214 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0d86e529-5bd4-42f9-b844-f5e06417b52a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:41.545731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0d86e529-5bd4-42f9-b844-f5e06417b52a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:41.553192 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0d86e529-5bd4-42f9-b844-f5e06417b52a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:41.554192 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0d86e529-5bd4-42f9-b844-f5e06417b52a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:41.554745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0d86e529-5bd4-42f9-b844-f5e06417b52a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:41.555407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0d86e529-5bd4-42f9-b844-f5e06417b52a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:41.555869 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0d86e529-5bd4-42f9-b844-f5e06417b52a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:41.556282 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0d86e529-5bd4-42f9-b844-f5e06417b52a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:41.698785 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0d86e529-5bd4-42f9-b844-f5e06417b52a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:41.708603 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0d86e529-5bd4-42f9-b844-f5e06417b52a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.167336 Oct 20 09:03:41.709411 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 226/474] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:41 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:42.313101 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:42.313925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:42.320788 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:42.321267 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:42.321704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:42.322396 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:42.322875 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:42.323297 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:42.466492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:42.477306 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:42.479638 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:42.481265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c9a55545-6bdb-42ff-a2bb-db00c50801ca tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.171870 Oct 20 09:03:42.481864 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 227/475] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:42 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:42.732337 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8a4ce115-02cc-4a77-8538-5b6ef64d98f4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:42.732904 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8a4ce115-02cc-4a77-8538-5b6ef64d98f4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:42.739922 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a4ce115-02cc-4a77-8538-5b6ef64d98f4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:42.740413 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a4ce115-02cc-4a77-8538-5b6ef64d98f4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:42.740881 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a4ce115-02cc-4a77-8538-5b6ef64d98f4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:42.742130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a4ce115-02cc-4a77-8538-5b6ef64d98f4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:42.742617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a4ce115-02cc-4a77-8538-5b6ef64d98f4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:42.743154 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8a4ce115-02cc-4a77-8538-5b6ef64d98f4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:42.885748 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8a4ce115-02cc-4a77-8538-5b6ef64d98f4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:42.894590 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8a4ce115-02cc-4a77-8538-5b6ef64d98f4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.166080 Oct 20 09:03:42.895355 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 228/476] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:42 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:43.508678 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=6 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:43.509266 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:43.510139 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:43.516108 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:43.516597 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:43.517069 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:43.517770 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:43.518213 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:43.518667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:43.676644 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:43.685555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:43.687026 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:43.688208 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-974b890b-5e8b-49f5-a1b4-af1a0c72fa69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.183011 Oct 20 09:03:43.688771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 229/477] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:43 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:43.917235 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-de0dbb97-9a3a-4fbc-92ac-3e4aa26f54c1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:43.918260 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-de0dbb97-9a3a-4fbc-92ac-3e4aa26f54c1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:43.924769 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de0dbb97-9a3a-4fbc-92ac-3e4aa26f54c1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:43.925326 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de0dbb97-9a3a-4fbc-92ac-3e4aa26f54c1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:43.925818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de0dbb97-9a3a-4fbc-92ac-3e4aa26f54c1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:43.926596 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de0dbb97-9a3a-4fbc-92ac-3e4aa26f54c1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:43.927190 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de0dbb97-9a3a-4fbc-92ac-3e4aa26f54c1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:43.927700 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de0dbb97-9a3a-4fbc-92ac-3e4aa26f54c1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:44.072666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-de0dbb97-9a3a-4fbc-92ac-3e4aa26f54c1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:44.080735 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-de0dbb97-9a3a-4fbc-92ac-3e4aa26f54c1 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.167353 Oct 20 09:03:44.081302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 230/478] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:43 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:44.714166 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:44.714942 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:44.721257 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:44.721745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:44.722178 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:44.722903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:44.723354 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:44.723779 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:44.863250 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:44.872161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:44.873617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:44.874885 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b456bef2-9f95-4e5d-a1e6-a28df75382d9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.165395 Oct 20 09:03:44.875453 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 231/479] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:44 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:45.103852 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-026a429d-1354-4a47-a26c-f9cd4f23501a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:45.104373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-026a429d-1354-4a47-a26c-f9cd4f23501a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:45.111922 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-026a429d-1354-4a47-a26c-f9cd4f23501a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:45.112407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-026a429d-1354-4a47-a26c-f9cd4f23501a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:45.112835 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-026a429d-1354-4a47-a26c-f9cd4f23501a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:45.113526 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-026a429d-1354-4a47-a26c-f9cd4f23501a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:45.113973 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-026a429d-1354-4a47-a26c-f9cd4f23501a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:45.114398 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-026a429d-1354-4a47-a26c-f9cd4f23501a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:45.246190 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-026a429d-1354-4a47-a26c-f9cd4f23501a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:45.253788 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-026a429d-1354-4a47-a26c-f9cd4f23501a tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.153975 Oct 20 09:03:45.254342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 232/480] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:45 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:45.901428 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:45.902088 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:45.907935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:45.908417 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:45.908852 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:45.909526 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:45.909972 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:45.910396 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:46.041943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:46.050448 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:46.051914 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:46.053065 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-90ad8cac-3588-4795-8b46-640c7dc4ee19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.156241 Oct 20 09:03:46.053619 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 233/481] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:45 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:46.277131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-39d209d1-4f03-41a3-b9d8-c054c9121506 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:46.277653 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-39d209d1-4f03-41a3-b9d8-c054c9121506 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:46.283990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-39d209d1-4f03-41a3-b9d8-c054c9121506 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:46.284461 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-39d209d1-4f03-41a3-b9d8-c054c9121506 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:46.284888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-39d209d1-4f03-41a3-b9d8-c054c9121506 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:46.285558 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-39d209d1-4f03-41a3-b9d8-c054c9121506 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:46.285998 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-39d209d1-4f03-41a3-b9d8-c054c9121506 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:46.286413 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-39d209d1-4f03-41a3-b9d8-c054c9121506 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:46.415863 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-39d209d1-4f03-41a3-b9d8-c054c9121506 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:46.425010 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-39d209d1-4f03-41a3-b9d8-c054c9121506 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.151556 Oct 20 09:03:46.441916 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 234/482] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:46 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:46.559906 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=90,nova_cell1:INSERT=4,nova_cell1:UPDATE=6 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:46.574278 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-502b6248-7e6a-4ef4-9b69-f7b954d22c59 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 8.603536 Oct 20 09:03:46.575591 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 249/483] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:03:37 2022] POST /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/os-volume_attachments => generated 194 bytes in 8606 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:03:47.077875 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:47.078536 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:47.085123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:47.086129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:47.086598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:47.087292 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:47.087747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:47.088179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:47.219785 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:47.228559 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:47.229991 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:47.231205 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c9f5de21-7a9a-4d1c-bb9b-3d765d6bc09e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.157252 Oct 20 09:03:47.231766 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 235/484] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:47 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:47.450670 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b2c14ccd-dcb7-4ad3-be83-01b3758286c4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:47.451367 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b2c14ccd-dcb7-4ad3-be83-01b3758286c4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:47.459757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b2c14ccd-dcb7-4ad3-be83-01b3758286c4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:47.460240 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b2c14ccd-dcb7-4ad3-be83-01b3758286c4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:47.460688 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b2c14ccd-dcb7-4ad3-be83-01b3758286c4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:47.461364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b2c14ccd-dcb7-4ad3-be83-01b3758286c4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:47.461814 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b2c14ccd-dcb7-4ad3-be83-01b3758286c4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:47.462237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b2c14ccd-dcb7-4ad3-be83-01b3758286c4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:47.595828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b2c14ccd-dcb7-4ad3-be83-01b3758286c4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:47.603553 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b2c14ccd-dcb7-4ad3-be83-01b3758286c4 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.157165 Oct 20 09:03:47.604117 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 250/485] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:47 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:48.255663 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:48.256388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:48.263687 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:48.264188 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:48.264610 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:48.265302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:48.265752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:48.266178 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:48.280977 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:48.397384 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:48.406044 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:48.407513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:48.408665 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-65376d3c-8fea-4c0f-b348-2e16362d3e8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.157049 Oct 20 09:03:48.409224 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 236/486] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:48 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:48.626460 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9850925a-bc4d-4f69-9689-fb71bcb4cf23 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:48.627088 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9850925a-bc4d-4f69-9689-fb71bcb4cf23 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:48.633444 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9850925a-bc4d-4f69-9689-fb71bcb4cf23 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:48.633923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9850925a-bc4d-4f69-9689-fb71bcb4cf23 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:48.634353 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9850925a-bc4d-4f69-9689-fb71bcb4cf23 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:48.635060 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9850925a-bc4d-4f69-9689-fb71bcb4cf23 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:48.635507 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9850925a-bc4d-4f69-9689-fb71bcb4cf23 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:48.635928 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9850925a-bc4d-4f69-9689-fb71bcb4cf23 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:48.767104 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9850925a-bc4d-4f69-9689-fb71bcb4cf23 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:48.774601 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9850925a-bc4d-4f69-9689-fb71bcb4cf23 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.151946 Oct 20 09:03:48.775164 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 251/487] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:48 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:49.436520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:49.437126 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:49.443669 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:49.444161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:49.444597 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:49.445307 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:49.445756 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:49.446210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:49.573014 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:49.581470 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:49.582928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:49.584069 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-591b1c81-7438-47bd-907a-a19b7406dee5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.151353 Oct 20 09:03:49.584624 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 237/488] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:49 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:49.796613 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ca1b7320-8da4-4d02-8a16-f640fe6c4e2b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:49.797211 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ca1b7320-8da4-4d02-8a16-f640fe6c4e2b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:49.799620 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=49,nova_api:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:03:49.804790 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ca1b7320-8da4-4d02-8a16-f640fe6c4e2b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:49.805326 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ca1b7320-8da4-4d02-8a16-f640fe6c4e2b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:49.806269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ca1b7320-8da4-4d02-8a16-f640fe6c4e2b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:49.807055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ca1b7320-8da4-4d02-8a16-f640fe6c4e2b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:49.807505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ca1b7320-8da4-4d02-8a16-f640fe6c4e2b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:49.807920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ca1b7320-8da4-4d02-8a16-f640fe6c4e2b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:49.936287 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ca1b7320-8da4-4d02-8a16-f640fe6c4e2b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:49.943854 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ca1b7320-8da4-4d02-8a16-f640fe6c4e2b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.151081 Oct 20 09:03:49.944411 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 252/489] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:49 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:50.609055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:50.609660 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:50.616094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:50.616572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:50.617005 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:50.617680 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:50.618125 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:50.618572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:50.747715 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:50.756265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:50.757695 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:50.758867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-848f3d9a-62f6-4131-95c2-76730828d3b5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.153642 Oct 20 09:03:50.759423 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 238/490] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:50 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:50.967162 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9b0292e4-ca19-4927-8bc1-a5ed36ed610e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:50.967918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9b0292e4-ca19-4927-8bc1-a5ed36ed610e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:50.974033 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9b0292e4-ca19-4927-8bc1-a5ed36ed610e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:50.974529 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9b0292e4-ca19-4927-8bc1-a5ed36ed610e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:50.974958 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9b0292e4-ca19-4927-8bc1-a5ed36ed610e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:50.975638 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9b0292e4-ca19-4927-8bc1-a5ed36ed610e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:50.976077 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9b0292e4-ca19-4927-8bc1-a5ed36ed610e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:50.976491 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9b0292e4-ca19-4927-8bc1-a5ed36ed610e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:51.106408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9b0292e4-ca19-4927-8bc1-a5ed36ed610e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:51.114185 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9b0292e4-ca19-4927-8bc1-a5ed36ed610e tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.150978 Oct 20 09:03:51.114883 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 253/491] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:50 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:51.195846 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-46b9d1c6-2024-46da-b2c4-9c81a9c19e33 req-def85d39-cd3a-43fc-a29b-04122eb00df0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "98592479-f7da-4df7-ad14-c6aea41d81e0", "name": "network-vif-plugged", "status": "completed", "tag": "eeeae5ec-ff15-4944-a558-e9f736d0ca1e"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:51.206496 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-46b9d1c6-2024-46da-b2c4-9c81a9c19e33 req-def85d39-cd3a-43fc-a29b-04122eb00df0 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:51.207633 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-46b9d1c6-2024-46da-b2c4-9c81a9c19e33 req-def85d39-cd3a-43fc-a29b-04122eb00df0 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:51.208212 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-46b9d1c6-2024-46da-b2c4-9c81a9c19e33 req-def85d39-cd3a-43fc-a29b-04122eb00df0 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:51.220166 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-46b9d1c6-2024-46da-b2c4-9c81a9c19e33 req-def85d39-cd3a-43fc-a29b-04122eb00df0 service nova] Creating event network-vif-plugged:eeeae5ec-ff15-4944-a558-e9f736d0ca1e for instance 98592479-f7da-4df7-ad14-c6aea41d81e0 on hv-cinder-maste Oct 20 09:03:51.225346 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-46b9d1c6-2024-46da-b2c4-9c81a9c19e33 req-def85d39-cd3a-43fc-a29b-04122eb00df0 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033521 Oct 20 09:03:51.226035 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 239/492] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:03:51.783062 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:51.783662 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:51.789607 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:51.790059 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:51.790493 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:51.791236 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:51.791678 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:51.792100 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:51.920906 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:51.929511 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:51.931088 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:51.932803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b1319e6f-7d33-4c54-9b2f-b7e697934dd2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.153594 Oct 20 09:03:51.933356 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 254/493] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:51 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:52.136476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a6108224-7dcf-45c4-81a1-65ccb0cfe183 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:52.137167 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a6108224-7dcf-45c4-81a1-65ccb0cfe183 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:52.143848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6108224-7dcf-45c4-81a1-65ccb0cfe183 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:52.144317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6108224-7dcf-45c4-81a1-65ccb0cfe183 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:52.144745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6108224-7dcf-45c4-81a1-65ccb0cfe183 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:52.145418 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6108224-7dcf-45c4-81a1-65ccb0cfe183 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:52.145862 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6108224-7dcf-45c4-81a1-65ccb0cfe183 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:52.146278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6108224-7dcf-45c4-81a1-65ccb0cfe183 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:52.276218 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a6108224-7dcf-45c4-81a1-65ccb0cfe183 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:52.283787 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a6108224-7dcf-45c4-81a1-65ccb0cfe183 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.151513 Oct 20 09:03:52.284345 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 240/494] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:52 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:52.958441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:52.959074 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:52.965722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:52.966210 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:52.966675 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:52.967398 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:52.967849 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:52.968280 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:53.099223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:53.108226 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:53.109735 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:53.110916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b635fd26-819c-44fd-a7ee-952a5a7c24eb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.156392 Oct 20 09:03:53.111476 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 255/495] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:52 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:53.306199 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8bf93497-a0b1-4404-8b3b-d9a5ea3c426b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:53.307471 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8bf93497-a0b1-4404-8b3b-d9a5ea3c426b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:53.313498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8bf93497-a0b1-4404-8b3b-d9a5ea3c426b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:53.313973 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8bf93497-a0b1-4404-8b3b-d9a5ea3c426b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:53.314400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8bf93497-a0b1-4404-8b3b-d9a5ea3c426b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:53.315113 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8bf93497-a0b1-4404-8b3b-d9a5ea3c426b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:53.315555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8bf93497-a0b1-4404-8b3b-d9a5ea3c426b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:53.315972 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8bf93497-a0b1-4404-8b3b-d9a5ea3c426b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:53.677616 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8bf93497-a0b1-4404-8b3b-d9a5ea3c426b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:53.685151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8bf93497-a0b1-4404-8b3b-d9a5ea3c426b tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1319 microversion: 2.1 time: 0.382819 Oct 20 09:03:53.685706 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 241/496] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:53 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1319 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:54.136801 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:54.137401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:54.143808 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:54.144284 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:54.144741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:54.145421 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:54.145869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:54.146290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:54.271707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:54.280474 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:54.281950 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:54.283127 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1ec994c3-8d8e-4c42-8a94-4d25d8c68fa4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.150791 Oct 20 09:03:54.283692 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 256/497] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:54 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:54.560967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1864329537", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1088348766", "security_groups": [{"name": "tempest-secgroup-smoke-1673307534"}], "block_device_mapping_v2": [{"uuid": "3e371462-e17e-4d91-b898-d08f40a65134", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "d2d4ae81-03f2-49c0-8ca3-a270479acf30"}]}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:54.673577 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] validate_networks() for [('d2d4ae81-03f2-49c0-8ca3-a270479acf30', None, None, None, None, None)] {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:03:54.708309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-28b74a7c-8791-4cec-b3af-d28bb0666017 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:54.708906 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-28b74a7c-8791-4cec-b3af-d28bb0666017 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:54.715885 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28b74a7c-8791-4cec-b3af-d28bb0666017 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:54.716360 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28b74a7c-8791-4cec-b3af-d28bb0666017 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:54.716786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28b74a7c-8791-4cec-b3af-d28bb0666017 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:54.717458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28b74a7c-8791-4cec-b3af-d28bb0666017 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:54.717902 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28b74a7c-8791-4cec-b3af-d28bb0666017 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:54.718316 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28b74a7c-8791-4cec-b3af-d28bb0666017 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:54.848488 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-28b74a7c-8791-4cec-b3af-d28bb0666017 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:54.857117 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-28b74a7c-8791-4cec-b3af-d28bb0666017 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1507 microversion: 2.1 time: 0.152603 Oct 20 09:03:54.857786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 257/498] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:54 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1507 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:54.879350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-aa809551-6187-4b3a-ab5e-2aeafb40527f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "18201c29-7bb8-495f-9606-f7fec1347fd1", "device": "/dev/sdb"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:54.882326 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-aa809551-6187-4b3a-ab5e-2aeafb40527f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:54.888215 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa809551-6187-4b3a-ab5e-2aeafb40527f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:54.888698 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa809551-6187-4b3a-ab5e-2aeafb40527f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:54.889132 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa809551-6187-4b3a-ab5e-2aeafb40527f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:54.889816 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa809551-6187-4b3a-ab5e-2aeafb40527f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:54.890268 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa809551-6187-4b3a-ab5e-2aeafb40527f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:54.890718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa809551-6187-4b3a-ab5e-2aeafb40527f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:55.042216 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:03:55.042682 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:03:55.043154 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:03:55.043560 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:03:55.043968 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:03:55.052700 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:55.053180 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:55.053753 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:55.054541 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:55.054997 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:55.055419 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:55.072439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Getting quotas for project 98827dbef155480b9f26792b70014b4b. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:03:55.076362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Getting quotas for user 65ca2eae6db24bb094afaacdfb5c1980 and project 98827dbef155480b9f26792b70014b4b. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:03:55.137222 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:03:55.139443 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='3e371462-e17e-4d91-b898-d08f40a65134',volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:03:55.278264 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-afb3cbc1-9357-442c-9b20-10ffe4eb147c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.722190 Oct 20 09:03:55.279231 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 242/499] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:03:54 2022] POST /compute/v2.1/servers => generated 406 bytes in 724 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:03:55.283580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-189f2d80-2cd1-4ada-a5a1-8fba4405bcdb req-089e7eb0-799c-43ba-b01f-2a3cb34a8fe5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f8d61907-24ca-4e74-917e-d218d0f89557", "name": "network-vif-plugged", "status": "completed", "tag": "6ba2009e-08be-4c65-a1d5-5e9548c8d407"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:55.293742 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-189f2d80-2cd1-4ada-a5a1-8fba4405bcdb req-089e7eb0-799c-43ba-b01f-2a3cb34a8fe5 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:55.294149 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-189f2d80-2cd1-4ada-a5a1-8fba4405bcdb req-089e7eb0-799c-43ba-b01f-2a3cb34a8fe5 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:55.294612 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-189f2d80-2cd1-4ada-a5a1-8fba4405bcdb req-089e7eb0-799c-43ba-b01f-2a3cb34a8fe5 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:55.305762 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-189f2d80-2cd1-4ada-a5a1-8fba4405bcdb req-089e7eb0-799c-43ba-b01f-2a3cb34a8fe5 service nova] Creating event network-vif-plugged:6ba2009e-08be-4c65-a1d5-5e9548c8d407 for instance f8d61907-24ca-4e74-917e-d218d0f89557 on hv-cinder-maste Oct 20 09:03:55.311253 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-189f2d80-2cd1-4ada-a5a1-8fba4405bcdb req-089e7eb0-799c-43ba-b01f-2a3cb34a8fe5 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031032 Oct 20 09:03:55.311850 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 243/500] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:03:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:03:55.316841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3a290408-2eba-43e0-893c-0fd41bc3f671 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:55.317454 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3a290408-2eba-43e0-893c-0fd41bc3f671 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:55.373234 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3a290408-2eba-43e0-893c-0fd41bc3f671 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:55.380369 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3a290408-2eba-43e0-893c-0fd41bc3f671 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1095 microversion: 2.1 time: 0.067246 Oct 20 09:03:55.380943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 244/501] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:55 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1095 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:55.385366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:55.385962 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:55.392816 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:55.393335 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:55.393776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:55.394510 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:55.394995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:55.395512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:55.537963 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:55.546536 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:55.547912 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:55.549053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-581908b2-32c6-4498-85bf-49cdb98a6fb4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.167209 Oct 20 09:03:55.549607 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 245/502] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:55 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:56.251255 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-aa809551-6187-4b3a-ab5e-2aeafb40527f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "POST /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.375584 Oct 20 09:03:56.252373 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 258/503] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:03:54 2022] POST /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0/os-volume_attachments => generated 194 bytes in 1377 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:03:56.266678 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6ec1c952-ee90-4aa9-ab82-3c3f44eb04f2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:03:56.273175 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ec1c952-ee90-4aa9-ab82-3c3f44eb04f2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:56.273735 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ec1c952-ee90-4aa9-ab82-3c3f44eb04f2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:56.274281 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ec1c952-ee90-4aa9-ab82-3c3f44eb04f2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:56.340036 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6ec1c952-ee90-4aa9-ab82-3c3f44eb04f2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Going to try to stop instance {{(pid=119068) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 20 09:03:56.396613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6ec1c952-ee90-4aa9-ab82-3c3f44eb04f2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/action" status: 202 len: 0 microversion: 2.1 time: 0.133692 Oct 20 09:03:56.400849 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 246/504] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:03:56 2022] POST /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/action => generated 0 bytes in 136 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:03:56.405425 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-82e9d7f9-299a-4672-b274-9c0eceb07541 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:56.406123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-82e9d7f9-299a-4672-b274-9c0eceb07541 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:56.421548 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82e9d7f9-299a-4672-b274-9c0eceb07541 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:56.422977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82e9d7f9-299a-4672-b274-9c0eceb07541 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:56.423568 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82e9d7f9-299a-4672-b274-9c0eceb07541 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:56.423684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b7f592bd-11b4-48a8-bec4-10f626ce847b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:56.424383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b7f592bd-11b4-48a8-bec4-10f626ce847b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:56.424592 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82e9d7f9-299a-4672-b274-9c0eceb07541 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:56.425210 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82e9d7f9-299a-4672-b274-9c0eceb07541 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:56.425726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-82e9d7f9-299a-4672-b274-9c0eceb07541 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:56.431035 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b7f592bd-11b4-48a8-bec4-10f626ce847b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:56.431683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b7f592bd-11b4-48a8-bec4-10f626ce847b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:56.432189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b7f592bd-11b4-48a8-bec4-10f626ce847b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:56.433013 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b7f592bd-11b4-48a8-bec4-10f626ce847b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:56.433569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b7f592bd-11b4-48a8-bec4-10f626ce847b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:56.434328 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b7f592bd-11b4-48a8-bec4-10f626ce847b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:56.490125 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-82e9d7f9-299a-4672-b274-9c0eceb07541 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:56.498013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-82e9d7f9-299a-4672-b274-9c0eceb07541 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1193 microversion: 2.1 time: 0.098443 Oct 20 09:03:56.498609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 259/505] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:56 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1193 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:56.568344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b7f592bd-11b4-48a8-bec4-10f626ce847b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:56.575653 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:56.576257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:56.576929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b7f592bd-11b4-48a8-bec4-10f626ce847b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.165954 Oct 20 09:03:56.577500 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 247/506] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:56 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:56.582249 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:56.582747 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:56.583183 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:56.583865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:56.584314 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:56.584731 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:56.711926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:56.721444 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:56.722919 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:56.724073 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-73ba5655-fc72-4095-aa0e-d8930f3d0d33 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.152585 Oct 20 09:03:56.724638 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 260/507] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:56 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:57.519860 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-03225940-01bb-4f7c-923d-27d944b65207 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:57.520462 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-03225940-01bb-4f7c-923d-27d944b65207 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:57.526930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-03225940-01bb-4f7c-923d-27d944b65207 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:57.527406 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-03225940-01bb-4f7c-923d-27d944b65207 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:57.527834 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-03225940-01bb-4f7c-923d-27d944b65207 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:57.528507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-03225940-01bb-4f7c-923d-27d944b65207 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:57.528949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-03225940-01bb-4f7c-923d-27d944b65207 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:57.529366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-03225940-01bb-4f7c-923d-27d944b65207 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:57.592437 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-03225940-01bb-4f7c-923d-27d944b65207 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:57.601302 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0dacf0ee-3312-4195-b227-858f94611803 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:57.601812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-03225940-01bb-4f7c-923d-27d944b65207 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1193 microversion: 2.1 time: 0.085693 Oct 20 09:03:57.602060 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0dacf0ee-3312-4195-b227-858f94611803 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:57.602376 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 248/508] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:57 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1193 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:57.609014 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0dacf0ee-3312-4195-b227-858f94611803 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:57.609558 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0dacf0ee-3312-4195-b227-858f94611803 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:57.610181 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0dacf0ee-3312-4195-b227-858f94611803 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:57.610940 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0dacf0ee-3312-4195-b227-858f94611803 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:57.611381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0dacf0ee-3312-4195-b227-858f94611803 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:57.611802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0dacf0ee-3312-4195-b227-858f94611803 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:57.742854 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0dacf0ee-3312-4195-b227-858f94611803 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:57.750747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:57.751582 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:57.752068 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0dacf0ee-3312-4195-b227-858f94611803 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.155019 Oct 20 09:03:57.752946 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 261/509] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:57 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:03:57.757744 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:57.758329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:57.758882 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:57.759683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:57.760232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:57.760718 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:57.887134 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:57.895687 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:57.897159 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:57.898568 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-45c404e5-0b61-4f4b-ba9a-79a833886df2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1805 microversion: 2.20 time: 0.151649 Oct 20 09:03:57.899294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 249/510] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:57 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1805 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:03:58.625645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b7eac92c-ee76-456c-aa81-a00efc315d9b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:58.626349 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b7eac92c-ee76-456c-aa81-a00efc315d9b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:58.633049 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b7eac92c-ee76-456c-aa81-a00efc315d9b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:58.633608 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b7eac92c-ee76-456c-aa81-a00efc315d9b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:58.633967 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b7eac92c-ee76-456c-aa81-a00efc315d9b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:58.635322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b7eac92c-ee76-456c-aa81-a00efc315d9b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:58.635775 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b7eac92c-ee76-456c-aa81-a00efc315d9b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:58.636211 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b7eac92c-ee76-456c-aa81-a00efc315d9b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:58.696714 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b7eac92c-ee76-456c-aa81-a00efc315d9b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:58.704196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b7eac92c-ee76-456c-aa81-a00efc315d9b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1193 microversion: 2.1 time: 0.082441 Oct 20 09:03:58.704757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 262/511] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:58 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1193 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:58.776450 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7bd61b6f-7f19-4def-8aa6-f5ad8f07f0bd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:58.777109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7bd61b6f-7f19-4def-8aa6-f5ad8f07f0bd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:58.783568 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bd61b6f-7f19-4def-8aa6-f5ad8f07f0bd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:58.784057 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bd61b6f-7f19-4def-8aa6-f5ad8f07f0bd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:58.784497 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bd61b6f-7f19-4def-8aa6-f5ad8f07f0bd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:58.785183 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bd61b6f-7f19-4def-8aa6-f5ad8f07f0bd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:58.785637 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bd61b6f-7f19-4def-8aa6-f5ad8f07f0bd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:58.786062 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7bd61b6f-7f19-4def-8aa6-f5ad8f07f0bd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:58.924646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7bd61b6f-7f19-4def-8aa6-f5ad8f07f0bd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:58.930680 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:58.931287 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:58.940162 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:58.940789 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:58.941314 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:58.942158 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:58.942771 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:58.943119 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7bd61b6f-7f19-4def-8aa6-f5ad8f07f0bd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.170730 Oct 20 09:03:58.943417 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:58.944049 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 250/512] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:58 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:03:59.077719 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:59.087183 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:59.089534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:59.091011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-80fe06e0-359e-49ca-ae07-43e05837ad74 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1792 microversion: 2.20 time: 0.169514 Oct 20 09:03:59.091572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 263/513] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:03:58 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1792 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:03:59.727583 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bc20a194-cb57-4991-8e52-7d10c180d484 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:59.728344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bc20a194-cb57-4991-8e52-7d10c180d484 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:59.737331 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc20a194-cb57-4991-8e52-7d10c180d484 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:59.738103 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc20a194-cb57-4991-8e52-7d10c180d484 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:59.738822 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc20a194-cb57-4991-8e52-7d10c180d484 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:59.739659 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc20a194-cb57-4991-8e52-7d10c180d484 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:59.740221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc20a194-cb57-4991-8e52-7d10c180d484 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:59.740752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc20a194-cb57-4991-8e52-7d10c180d484 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:59.968156 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f0c7354c-a148-4d6f-93f3-eee66b38fa00 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:03:59.969474 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f0c7354c-a148-4d6f-93f3-eee66b38fa00 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:03:59.970458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bc20a194-cb57-4991-8e52-7d10c180d484 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:03:59.976997 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0c7354c-a148-4d6f-93f3-eee66b38fa00 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:59.977837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0c7354c-a148-4d6f-93f3-eee66b38fa00 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:59.978439 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0c7354c-a148-4d6f-93f3-eee66b38fa00 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:59.979924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0c7354c-a148-4d6f-93f3-eee66b38fa00 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:03:59.980374 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0c7354c-a148-4d6f-93f3-eee66b38fa00 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:03:59.981092 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f0c7354c-a148-4d6f-93f3-eee66b38fa00 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:03:59.985364 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bc20a194-cb57-4991-8e52-7d10c180d484 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1279 microversion: 2.1 time: 0.261152 Oct 20 09:03:59.987748 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 251/514] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:59 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1279 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:00.121222 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f0c7354c-a148-4d6f-93f3-eee66b38fa00 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:00.129375 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f0c7354c-a148-4d6f-93f3-eee66b38fa00 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.165703 Oct 20 09:04:00.130029 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 264/515] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:03:59 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:00.498632 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-3c896d05-a648-462e-951e-299e016f7ba3 req-10e8ae76-1d10-4a5b-8f07-78a8d2b05493 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4b1d0b97-f53e-40dc-895f-48be8de838ee", "tag": "ea6a461f-d2ae-49cf-a87f-19296d8271f9"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:00.509481 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-3c896d05-a648-462e-951e-299e016f7ba3 req-10e8ae76-1d10-4a5b-8f07-78a8d2b05493 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:00.510508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-3c896d05-a648-462e-951e-299e016f7ba3 req-10e8ae76-1d10-4a5b-8f07-78a8d2b05493 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:00.511020 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-3c896d05-a648-462e-951e-299e016f7ba3 req-10e8ae76-1d10-4a5b-8f07-78a8d2b05493 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:00.523020 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-3c896d05-a648-462e-951e-299e016f7ba3 req-10e8ae76-1d10-4a5b-8f07-78a8d2b05493 service nova] Creating event network-changed:ea6a461f-d2ae-49cf-a87f-19296d8271f9 for instance 4b1d0b97-f53e-40dc-895f-48be8de838ee on hv-cinder-maste Oct 20 09:04:00.528324 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-3c896d05-a648-462e-951e-299e016f7ba3 req-10e8ae76-1d10-4a5b-8f07-78a8d2b05493 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034361 Oct 20 09:04:00.529235 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 252/516] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:01.009247 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c511aeab-2236-4292-ae77-9aa315725d16 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:01.009820 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c511aeab-2236-4292-ae77-9aa315725d16 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:01.016247 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c511aeab-2236-4292-ae77-9aa315725d16 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:01.016725 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c511aeab-2236-4292-ae77-9aa315725d16 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:01.017161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c511aeab-2236-4292-ae77-9aa315725d16 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:01.017846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c511aeab-2236-4292-ae77-9aa315725d16 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:01.018294 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c511aeab-2236-4292-ae77-9aa315725d16 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:01.018741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c511aeab-2236-4292-ae77-9aa315725d16 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:01.022997 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:01.154139 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a4b79e73-e7d8-4168-8607-8ccd2738b23b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:01.155446 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a4b79e73-e7d8-4168-8607-8ccd2738b23b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:01.156116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c511aeab-2236-4292-ae77-9aa315725d16 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:01.161794 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a4b79e73-e7d8-4168-8607-8ccd2738b23b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:01.162341 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a4b79e73-e7d8-4168-8607-8ccd2738b23b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:01.162880 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a4b79e73-e7d8-4168-8607-8ccd2738b23b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:01.163568 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a4b79e73-e7d8-4168-8607-8ccd2738b23b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:01.164016 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a4b79e73-e7d8-4168-8607-8ccd2738b23b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:01.164454 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a4b79e73-e7d8-4168-8607-8ccd2738b23b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:01.165204 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c511aeab-2236-4292-ae77-9aa315725d16 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1279 microversion: 2.1 time: 0.159825 Oct 20 09:04:01.166458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 265/517] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:01 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1279 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:01.360234 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a4b79e73-e7d8-4168-8607-8ccd2738b23b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:01.368701 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a4b79e73-e7d8-4168-8607-8ccd2738b23b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.219258 Oct 20 09:04:01.369986 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 253/518] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:01 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:02.188795 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4e387d66-b33f-43ac-9377-acc227bf804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:02.189392 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4e387d66-b33f-43ac-9377-acc227bf804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:02.195772 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e387d66-b33f-43ac-9377-acc227bf804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:02.196243 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e387d66-b33f-43ac-9377-acc227bf804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:02.196665 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e387d66-b33f-43ac-9377-acc227bf804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:02.197867 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e387d66-b33f-43ac-9377-acc227bf804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:02.198315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e387d66-b33f-43ac-9377-acc227bf804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:02.198786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e387d66-b33f-43ac-9377-acc227bf804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:02.329241 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4e387d66-b33f-43ac-9377-acc227bf804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:02.336757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4e387d66-b33f-43ac-9377-acc227bf804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1279 microversion: 2.1 time: 0.151819 Oct 20 09:04:02.337311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 266/519] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:02 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1279 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:02.392045 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-89ace5ef-b69c-4e48-9b69-7b240c2f2fc0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:02.392679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-89ace5ef-b69c-4e48-9b69-7b240c2f2fc0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:02.399042 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89ace5ef-b69c-4e48-9b69-7b240c2f2fc0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:02.399528 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89ace5ef-b69c-4e48-9b69-7b240c2f2fc0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:02.399963 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89ace5ef-b69c-4e48-9b69-7b240c2f2fc0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:02.400738 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89ace5ef-b69c-4e48-9b69-7b240c2f2fc0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:02.401217 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89ace5ef-b69c-4e48-9b69-7b240c2f2fc0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:02.401641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89ace5ef-b69c-4e48-9b69-7b240c2f2fc0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:02.531284 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-89ace5ef-b69c-4e48-9b69-7b240c2f2fc0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:02.539422 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-89ace5ef-b69c-4e48-9b69-7b240c2f2fc0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.151306 Oct 20 09:04:02.539987 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 254/520] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:02 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:03.360512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3459b24f-93fc-4868-a43d-c75ef7555149 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:03.361135 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3459b24f-93fc-4868-a43d-c75ef7555149 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:03.367605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3459b24f-93fc-4868-a43d-c75ef7555149 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:03.368079 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3459b24f-93fc-4868-a43d-c75ef7555149 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:03.368507 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3459b24f-93fc-4868-a43d-c75ef7555149 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:03.369228 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3459b24f-93fc-4868-a43d-c75ef7555149 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:03.369660 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3459b24f-93fc-4868-a43d-c75ef7555149 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:03.370074 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3459b24f-93fc-4868-a43d-c75ef7555149 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:03.501999 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3459b24f-93fc-4868-a43d-c75ef7555149 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:03.510106 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3459b24f-93fc-4868-a43d-c75ef7555149 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1279 microversion: 2.1 time: 0.152952 Oct 20 09:04:03.510748 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 267/521] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:03 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1279 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:03.562674 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bcf2ae44-ee38-4710-b956-13fe08d04241 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:03.563286 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bcf2ae44-ee38-4710-b956-13fe08d04241 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:03.565805 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=3 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:03.571150 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bcf2ae44-ee38-4710-b956-13fe08d04241 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:03.571638 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bcf2ae44-ee38-4710-b956-13fe08d04241 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:03.572076 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bcf2ae44-ee38-4710-b956-13fe08d04241 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:03.573265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bcf2ae44-ee38-4710-b956-13fe08d04241 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:03.573721 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bcf2ae44-ee38-4710-b956-13fe08d04241 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:03.574147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bcf2ae44-ee38-4710-b956-13fe08d04241 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:03.699890 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bcf2ae44-ee38-4710-b956-13fe08d04241 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:03.702050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:03.709806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bcf2ae44-ee38-4710-b956-13fe08d04241 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.150882 Oct 20 09:04:03.710380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 255/522] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:03 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:04.532500 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-35c5ff11-24d2-4994-90e2-c39dc6ae32eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:04.533103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-35c5ff11-24d2-4994-90e2-c39dc6ae32eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:04.540117 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35c5ff11-24d2-4994-90e2-c39dc6ae32eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:04.540596 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35c5ff11-24d2-4994-90e2-c39dc6ae32eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:04.541527 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35c5ff11-24d2-4994-90e2-c39dc6ae32eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:04.542206 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35c5ff11-24d2-4994-90e2-c39dc6ae32eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:04.542715 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35c5ff11-24d2-4994-90e2-c39dc6ae32eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:04.543133 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35c5ff11-24d2-4994-90e2-c39dc6ae32eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:04.679987 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-35c5ff11-24d2-4994-90e2-c39dc6ae32eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:04.687669 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-35c5ff11-24d2-4994-90e2-c39dc6ae32eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1279 microversion: 2.1 time: 0.159005 Oct 20 09:04:04.688228 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 268/523] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:04 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1279 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:04.733818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-dca02a33-2c22-45d2-9b60-148410a34239 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:04.734415 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-dca02a33-2c22-45d2-9b60-148410a34239 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:04.741679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dca02a33-2c22-45d2-9b60-148410a34239 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:04.742151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dca02a33-2c22-45d2-9b60-148410a34239 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:04.742618 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dca02a33-2c22-45d2-9b60-148410a34239 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:04.743324 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dca02a33-2c22-45d2-9b60-148410a34239 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:04.743768 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dca02a33-2c22-45d2-9b60-148410a34239 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:04.744183 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dca02a33-2c22-45d2-9b60-148410a34239 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:04.870936 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-dca02a33-2c22-45d2-9b60-148410a34239 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:04.879067 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-dca02a33-2c22-45d2-9b60-148410a34239 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.149333 Oct 20 09:04:04.879638 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 256/524] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:04 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:05.709000 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-db56a428-375e-4fbe-9457-6af0ad282357 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:05.709598 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-db56a428-375e-4fbe-9457-6af0ad282357 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:05.716331 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db56a428-375e-4fbe-9457-6af0ad282357 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:05.716804 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db56a428-375e-4fbe-9457-6af0ad282357 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:05.717232 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db56a428-375e-4fbe-9457-6af0ad282357 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:05.717902 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db56a428-375e-4fbe-9457-6af0ad282357 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:05.718345 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db56a428-375e-4fbe-9457-6af0ad282357 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:05.718802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db56a428-375e-4fbe-9457-6af0ad282357 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:05.845941 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-db56a428-375e-4fbe-9457-6af0ad282357 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:05.854309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-db56a428-375e-4fbe-9457-6af0ad282357 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1279 microversion: 2.1 time: 0.149087 Oct 20 09:04:05.854899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 269/525] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:05 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1279 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:05.903175 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-498413bc-f424-4257-b34c-c66c427eede4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:05.903869 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-498413bc-f424-4257-b34c-c66c427eede4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:05.905764 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:05.906607 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=8 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:05.913035 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-498413bc-f424-4257-b34c-c66c427eede4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:05.913521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-498413bc-f424-4257-b34c-c66c427eede4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:05.913963 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-498413bc-f424-4257-b34c-c66c427eede4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:05.914856 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-498413bc-f424-4257-b34c-c66c427eede4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:05.915258 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-498413bc-f424-4257-b34c-c66c427eede4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:05.915686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-498413bc-f424-4257-b34c-c66c427eede4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:05.965891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 20 09:04:05.966103 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Thu Oct 20 09:04:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 (ip 10.222.0.36) !!! Oct 20 09:04:06.068732 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-498413bc-f424-4257-b34c-c66c427eede4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:06.077075 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-498413bc-f424-4257-b34c-c66c427eede4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.178160 Oct 20 09:04:06.077646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 257/526] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:05 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:06.877345 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-27f36a1c-ea56-452a-aa58-a897360ccf67 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:06.877970 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-27f36a1c-ea56-452a-aa58-a897360ccf67 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:06.884369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27f36a1c-ea56-452a-aa58-a897360ccf67 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:06.885394 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27f36a1c-ea56-452a-aa58-a897360ccf67 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:06.885822 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27f36a1c-ea56-452a-aa58-a897360ccf67 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:06.886519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27f36a1c-ea56-452a-aa58-a897360ccf67 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:06.886968 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27f36a1c-ea56-452a-aa58-a897360ccf67 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:06.887387 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27f36a1c-ea56-452a-aa58-a897360ccf67 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:07.016263 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-27f36a1c-ea56-452a-aa58-a897360ccf67 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:07.023803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-27f36a1c-ea56-452a-aa58-a897360ccf67 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.150311 Oct 20 09:04:07.024369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 270/527] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:06 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:07.101329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ca2cbed7-6f55-4fbf-b70c-3a2f2f83e8dd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:07.101932 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ca2cbed7-6f55-4fbf-b70c-3a2f2f83e8dd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:07.108239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca2cbed7-6f55-4fbf-b70c-3a2f2f83e8dd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:07.108714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca2cbed7-6f55-4fbf-b70c-3a2f2f83e8dd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:07.109142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca2cbed7-6f55-4fbf-b70c-3a2f2f83e8dd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:07.109814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca2cbed7-6f55-4fbf-b70c-3a2f2f83e8dd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:07.110257 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca2cbed7-6f55-4fbf-b70c-3a2f2f83e8dd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:07.110702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca2cbed7-6f55-4fbf-b70c-3a2f2f83e8dd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:07.239289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ca2cbed7-6f55-4fbf-b70c-3a2f2f83e8dd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:07.248838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ca2cbed7-6f55-4fbf-b70c-3a2f2f83e8dd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.151212 Oct 20 09:04:07.249441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 258/528] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:07 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:08.046573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f04055ef-056d-4573-b48d-8faaa79b804b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:08.047164 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f04055ef-056d-4573-b48d-8faaa79b804b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:08.053514 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f04055ef-056d-4573-b48d-8faaa79b804b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:08.053993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f04055ef-056d-4573-b48d-8faaa79b804b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:08.054426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f04055ef-056d-4573-b48d-8faaa79b804b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:08.055149 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f04055ef-056d-4573-b48d-8faaa79b804b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:08.055601 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f04055ef-056d-4573-b48d-8faaa79b804b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:08.056025 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f04055ef-056d-4573-b48d-8faaa79b804b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:08.188320 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f04055ef-056d-4573-b48d-8faaa79b804b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:08.195927 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f04055ef-056d-4573-b48d-8faaa79b804b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.153203 Oct 20 09:04:08.196490 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 271/529] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:08 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:08.272427 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fd5c3db4-252f-49a7-8585-90cb1ad2c502 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:08.273061 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-fd5c3db4-252f-49a7-8585-90cb1ad2c502 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:08.280056 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fd5c3db4-252f-49a7-8585-90cb1ad2c502 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:08.280534 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fd5c3db4-252f-49a7-8585-90cb1ad2c502 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:08.280969 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fd5c3db4-252f-49a7-8585-90cb1ad2c502 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:08.281651 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fd5c3db4-252f-49a7-8585-90cb1ad2c502 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:08.282526 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fd5c3db4-252f-49a7-8585-90cb1ad2c502 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:08.282526 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fd5c3db4-252f-49a7-8585-90cb1ad2c502 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:08.416897 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-fd5c3db4-252f-49a7-8585-90cb1ad2c502 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:08.424965 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fd5c3db4-252f-49a7-8585-90cb1ad2c502 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.156432 Oct 20 09:04:08.425530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 259/530] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:08 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:09.218833 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9ca3a99c-d5d0-45ac-95d8-53753fa2921c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:09.219455 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9ca3a99c-d5d0-45ac-95d8-53753fa2921c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:09.225887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca3a99c-d5d0-45ac-95d8-53753fa2921c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:09.226364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca3a99c-d5d0-45ac-95d8-53753fa2921c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:09.226827 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca3a99c-d5d0-45ac-95d8-53753fa2921c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:09.227508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca3a99c-d5d0-45ac-95d8-53753fa2921c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:09.227957 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca3a99c-d5d0-45ac-95d8-53753fa2921c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:09.229001 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9ca3a99c-d5d0-45ac-95d8-53753fa2921c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:09.355533 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9ca3a99c-d5d0-45ac-95d8-53753fa2921c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:09.363193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9ca3a99c-d5d0-45ac-95d8-53753fa2921c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.148447 Oct 20 09:04:09.363760 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 272/531] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:09 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:09.448720 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8568c51f-3fe2-433e-a6f6-f770b1a69f6a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:09.449318 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8568c51f-3fe2-433e-a6f6-f770b1a69f6a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:09.455700 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8568c51f-3fe2-433e-a6f6-f770b1a69f6a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:09.456176 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8568c51f-3fe2-433e-a6f6-f770b1a69f6a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:09.456605 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8568c51f-3fe2-433e-a6f6-f770b1a69f6a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:09.457296 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8568c51f-3fe2-433e-a6f6-f770b1a69f6a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:09.457739 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8568c51f-3fe2-433e-a6f6-f770b1a69f6a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:09.458178 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8568c51f-3fe2-433e-a6f6-f770b1a69f6a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:09.583706 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8568c51f-3fe2-433e-a6f6-f770b1a69f6a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:09.592964 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8568c51f-3fe2-433e-a6f6-f770b1a69f6a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.148635 Oct 20 09:04:09.594315 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 260/532] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:09 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:10.008995 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-cfc0e2ff-0206-445c-bc7d-35d6679ed242 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:10.010081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-cfc0e2ff-0206-445c-bc7d-35d6679ed242 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:10.015978 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cfc0e2ff-0206-445c-bc7d-35d6679ed242 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.016459 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cfc0e2ff-0206-445c-bc7d-35d6679ed242 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.016893 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cfc0e2ff-0206-445c-bc7d-35d6679ed242 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.017572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cfc0e2ff-0206-445c-bc7d-35d6679ed242 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.018017 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cfc0e2ff-0206-445c-bc7d-35d6679ed242 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.018436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cfc0e2ff-0206-445c-bc7d-35d6679ed242 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.040415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-cfc0e2ff-0206-445c-bc7d-35d6679ed242 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00/ips" status: 200 len: 150 microversion: 2.1 time: 0.035346 Oct 20 09:04:10.040962 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 273/533] 10.222.0.36 () {58 vars in 1248 bytes} [Thu Oct 20 09:04:10 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00/ips => generated 150 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:10.052447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "73bd08b5-9c2e-40b7-92db-80f1886711b9"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:10.054789 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:10.061340 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.061822 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.062256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.062980 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.063466 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.063885 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.142030 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Invalid volume: volume 73bd08b5-9c2e-40b7-92db-80f1886711b9 is already attached to instances: ec9ae57d-3e5a-490f-beba-d071e86a7b31 Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: POST /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00/os-volume_attachments HTTP/1.1 Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 74 Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:04:10.143868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {"volumeAttachment": {"volumeId": "73bd08b5-9c2e-40b7-92db-80f1886711b9"}} Oct 20 09:04:10.145191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Invalid volume: volume 73bd08b5-9c2e-40b7-92db-80f1886711b9 is already attached to instances: ec9ae57d-3e5a-490f-beba-d071e86a7b31 Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'server_id': 'aae7b08a-4c9a-4d12-9598-193afab4ca00', 'body': {'volumeAttachment': {'volumeId': '73bd08b5-9c2e-40b7-92db-80f1886711b9'}}} Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 73bd08b5-9c2e-40b7-92db-80f1886711b9 is already attached to instances: ec9ae57d-3e5a-490f-beba-d071e86a7b31 Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:10.145941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 73bd08b5-9c2e-40b7-92db-80f1886711b9 is already attached to instances: ec9ae57d-3e5a-490f-beba-d071e86a7b31 Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:10.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:04:10.148449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:04:10.148449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:04:10.148449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:04:10.148449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 73bd08b5-9c2e-40b7-92db-80f1886711b9 is already attached to instances: ec9ae57d-3e5a-490f-beba-d071e86a7b31 Oct 20 09:04:10.148449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:10.148449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 400 to user: Invalid volume: volume 73bd08b5-9c2e-40b7-92db-80f1886711b9 is already attached to instances: ec9ae57d-3e5a-490f-beba-d071e86a7b31 {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:04:10.148449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1690e904-a0ff-4736-88f5-91dc5f7437a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.096957 Oct 20 09:04:10.148449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 261/534] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:04:10 2022] POST /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00/os-volume_attachments => generated 174 bytes in 98 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 20 09:04:10.158649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-49e5ea25-c200-47aa-8e0f-201f8551bebb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:10.159148 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-49e5ea25-c200-47aa-8e0f-201f8551bebb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:10.165363 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-49e5ea25-c200-47aa-8e0f-201f8551bebb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.166465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-49e5ea25-c200-47aa-8e0f-201f8551bebb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.166859 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-49e5ea25-c200-47aa-8e0f-201f8551bebb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.167553 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-49e5ea25-c200-47aa-8e0f-201f8551bebb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.168004 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-49e5ea25-c200-47aa-8e0f-201f8551bebb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.168443 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-49e5ea25-c200-47aa-8e0f-201f8551bebb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.195941 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-49e5ea25-c200-47aa-8e0f-201f8551bebb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:04:10.276457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-49e5ea25-c200-47aa-8e0f-201f8551bebb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 204 len: 0 microversion: 2.1 time: 0.121319 Oct 20 09:04:10.278141 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 274/535] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:04:10 2022] DELETE /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:04:10.287260 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2e9eef68-ef1f-4854-8ca7-dcbcaa2358f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:10.287854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2e9eef68-ef1f-4854-8ca7-dcbcaa2358f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:10.295614 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2e9eef68-ef1f-4854-8ca7-dcbcaa2358f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.295955 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2e9eef68-ef1f-4854-8ca7-dcbcaa2358f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.296408 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2e9eef68-ef1f-4854-8ca7-dcbcaa2358f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.297102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2e9eef68-ef1f-4854-8ca7-dcbcaa2358f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.297613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2e9eef68-ef1f-4854-8ca7-dcbcaa2358f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.298084 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2e9eef68-ef1f-4854-8ca7-dcbcaa2358f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.385698 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-683c8d41-d674-47ef-9ab6-bd0652b9372b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:10.386397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-683c8d41-d674-47ef-9ab6-bd0652b9372b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:10.392415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-683c8d41-d674-47ef-9ab6-bd0652b9372b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.392899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-683c8d41-d674-47ef-9ab6-bd0652b9372b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.393337 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-683c8d41-d674-47ef-9ab6-bd0652b9372b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.394026 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-683c8d41-d674-47ef-9ab6-bd0652b9372b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.394493 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-683c8d41-d674-47ef-9ab6-bd0652b9372b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.394940 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-683c8d41-d674-47ef-9ab6-bd0652b9372b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.435200 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2e9eef68-ef1f-4854-8ca7-dcbcaa2358f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:10.443094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2e9eef68-ef1f-4854-8ca7-dcbcaa2358f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1681 microversion: 2.1 time: 0.160484 Oct 20 09:04:10.443756 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 262/536] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:10 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1681 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:10.527760 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-683c8d41-d674-47ef-9ab6-bd0652b9372b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:10.535281 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-683c8d41-d674-47ef-9ab6-bd0652b9372b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.153854 Oct 20 09:04:10.535843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 275/537] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:10 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:10.615317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f8566a36-bc6c-408c-a882-0cfe2f3971a6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:10.615949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f8566a36-bc6c-408c-a882-0cfe2f3971a6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:10.622311 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8566a36-bc6c-408c-a882-0cfe2f3971a6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.622819 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8566a36-bc6c-408c-a882-0cfe2f3971a6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.623256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8566a36-bc6c-408c-a882-0cfe2f3971a6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.623934 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8566a36-bc6c-408c-a882-0cfe2f3971a6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:10.624386 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8566a36-bc6c-408c-a882-0cfe2f3971a6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:10.624811 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8566a36-bc6c-408c-a882-0cfe2f3971a6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:10.750119 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f8566a36-bc6c-408c-a882-0cfe2f3971a6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:10.758196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f8566a36-bc6c-408c-a882-0cfe2f3971a6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.146688 Oct 20 09:04:10.758796 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 263/538] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:10 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:11.071140 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1e3f7345-5c5f-4fbf-9d69-315331cfb3d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "408e1832-d211-409c-a48e-dd0ec66831c0"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:11.074252 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1e3f7345-5c5f-4fbf-9d69-315331cfb3d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:11.080276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1e3f7345-5c5f-4fbf-9d69-315331cfb3d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:11.080763 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1e3f7345-5c5f-4fbf-9d69-315331cfb3d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:11.081202 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1e3f7345-5c5f-4fbf-9d69-315331cfb3d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:11.081890 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1e3f7345-5c5f-4fbf-9d69-315331cfb3d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:11.082367 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1e3f7345-5c5f-4fbf-9d69-315331cfb3d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:11.082825 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1e3f7345-5c5f-4fbf-9d69-315331cfb3d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:11.467106 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-79a333b7-4225-4ee8-95f1-75a306479baf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:11.467769 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-79a333b7-4225-4ee8-95f1-75a306479baf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:11.473777 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-79a333b7-4225-4ee8-95f1-75a306479baf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:11.474260 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-79a333b7-4225-4ee8-95f1-75a306479baf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:11.474757 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-79a333b7-4225-4ee8-95f1-75a306479baf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:11.475443 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-79a333b7-4225-4ee8-95f1-75a306479baf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:11.475894 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-79a333b7-4225-4ee8-95f1-75a306479baf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:11.476317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-79a333b7-4225-4ee8-95f1-75a306479baf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:11.603839 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-79a333b7-4225-4ee8-95f1-75a306479baf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:11.611753 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-79a333b7-4225-4ee8-95f1-75a306479baf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1681 microversion: 2.1 time: 0.148743 Oct 20 09:04:11.612317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 264/539] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:11 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1681 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:11.616600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-31103bde-daca-4cbf-ba64-8383850c4e18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:11.617195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-31103bde-daca-4cbf-ba64-8383850c4e18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:11.623584 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31103bde-daca-4cbf-ba64-8383850c4e18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:11.624062 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31103bde-daca-4cbf-ba64-8383850c4e18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:11.624495 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31103bde-daca-4cbf-ba64-8383850c4e18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:11.625177 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31103bde-daca-4cbf-ba64-8383850c4e18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:11.625630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31103bde-daca-4cbf-ba64-8383850c4e18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:11.626054 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-31103bde-daca-4cbf-ba64-8383850c4e18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:11.756650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-31103bde-daca-4cbf-ba64-8383850c4e18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:11.764527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-31103bde-daca-4cbf-ba64-8383850c4e18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.151275 Oct 20 09:04:11.765122 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 265/540] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:11 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:11.780545 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b46adde3-3832-422e-abde-6ff76da4d9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:11.781146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b46adde3-3832-422e-abde-6ff76da4d9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:11.786766 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b46adde3-3832-422e-abde-6ff76da4d9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:11.787249 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b46adde3-3832-422e-abde-6ff76da4d9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:11.787683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b46adde3-3832-422e-abde-6ff76da4d9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:11.788366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b46adde3-3832-422e-abde-6ff76da4d9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:11.788816 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b46adde3-3832-422e-abde-6ff76da4d9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:11.789240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b46adde3-3832-422e-abde-6ff76da4d9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:11.916094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b46adde3-3832-422e-abde-6ff76da4d9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:11.924250 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b46adde3-3832-422e-abde-6ff76da4d9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.147215 Oct 20 09:04:11.924812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 266/541] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:11 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:12.635738 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-80780d63-aba8-4ac7-9722-f6b808d9f920 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:12.636343 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-80780d63-aba8-4ac7-9722-f6b808d9f920 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:12.642643 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-80780d63-aba8-4ac7-9722-f6b808d9f920 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:12.643118 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-80780d63-aba8-4ac7-9722-f6b808d9f920 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:12.643546 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-80780d63-aba8-4ac7-9722-f6b808d9f920 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:12.644221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-80780d63-aba8-4ac7-9722-f6b808d9f920 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:12.644662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-80780d63-aba8-4ac7-9722-f6b808d9f920 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:12.645079 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-80780d63-aba8-4ac7-9722-f6b808d9f920 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:12.771043 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-80780d63-aba8-4ac7-9722-f6b808d9f920 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:12.779151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-80780d63-aba8-4ac7-9722-f6b808d9f920 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1681 microversion: 2.1 time: 0.147904 Oct 20 09:04:12.779714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 267/542] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:12 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1681 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:12.786366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4a3cb76a-689a-40c0-a203-a88e898bb8bf tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:12.786994 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4a3cb76a-689a-40c0-a203-a88e898bb8bf tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:12.792540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4a3cb76a-689a-40c0-a203-a88e898bb8bf tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:12.793015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4a3cb76a-689a-40c0-a203-a88e898bb8bf tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:12.793444 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4a3cb76a-689a-40c0-a203-a88e898bb8bf tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:12.794118 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4a3cb76a-689a-40c0-a203-a88e898bb8bf tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:12.794583 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4a3cb76a-689a-40c0-a203-a88e898bb8bf tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:12.795001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4a3cb76a-689a-40c0-a203-a88e898bb8bf tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:12.921278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4a3cb76a-689a-40c0-a203-a88e898bb8bf tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:12.928802 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4a3cb76a-689a-40c0-a203-a88e898bb8bf tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.146071 Oct 20 09:04:12.929358 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 268/543] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:12 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:12.948058 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8024f1d6-1d7a-4b14-ba04-bbf885d17378 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:12.948665 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8024f1d6-1d7a-4b14-ba04-bbf885d17378 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:12.954358 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8024f1d6-1d7a-4b14-ba04-bbf885d17378 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:12.954943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8024f1d6-1d7a-4b14-ba04-bbf885d17378 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:12.955383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8024f1d6-1d7a-4b14-ba04-bbf885d17378 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:12.956066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8024f1d6-1d7a-4b14-ba04-bbf885d17378 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:12.956518 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8024f1d6-1d7a-4b14-ba04-bbf885d17378 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:12.956944 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8024f1d6-1d7a-4b14-ba04-bbf885d17378 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:13.081639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8024f1d6-1d7a-4b14-ba04-bbf885d17378 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:13.090594 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8024f1d6-1d7a-4b14-ba04-bbf885d17378 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.145987 Oct 20 09:04:13.091157 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 269/544] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:12 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:13.801754 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-82b9c4c1-505d-41a9-a16e-b80c0e955412 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:13.802443 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-82b9c4c1-505d-41a9-a16e-b80c0e955412 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:13.809518 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-82b9c4c1-505d-41a9-a16e-b80c0e955412 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:13.810046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-82b9c4c1-505d-41a9-a16e-b80c0e955412 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:13.810501 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-82b9c4c1-505d-41a9-a16e-b80c0e955412 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:13.811381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-82b9c4c1-505d-41a9-a16e-b80c0e955412 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:13.811840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-82b9c4c1-505d-41a9-a16e-b80c0e955412 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:13.812265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-82b9c4c1-505d-41a9-a16e-b80c0e955412 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:13.814774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1e3f7345-5c5f-4fbf-9d69-315331cfb3d0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.747470 Oct 20 09:04:13.815501 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 276/545] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:04:11 2022] POST /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments => generated 194 bytes in 2749 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:13.944695 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-82b9c4c1-505d-41a9-a16e-b80c0e955412 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:13.952072 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-27c967dc-56cc-42ee-8ce9-24012589102c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:13.952809 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-27c967dc-56cc-42ee-8ce9-24012589102c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:13.952941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-82b9c4c1-505d-41a9-a16e-b80c0e955412 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1681 microversion: 2.1 time: 0.155439 Oct 20 09:04:13.953410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 270/546] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:13 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1681 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:13.959631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c967dc-56cc-42ee-8ce9-24012589102c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:13.960121 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c967dc-56cc-42ee-8ce9-24012589102c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:13.960557 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c967dc-56cc-42ee-8ce9-24012589102c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:13.961275 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c967dc-56cc-42ee-8ce9-24012589102c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:13.961729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c967dc-56cc-42ee-8ce9-24012589102c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:13.962182 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-27c967dc-56cc-42ee-8ce9-24012589102c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:14.092486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-27c967dc-56cc-42ee-8ce9-24012589102c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:14.100063 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-27c967dc-56cc-42ee-8ce9-24012589102c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.152180 Oct 20 09:04:14.100630 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 277/547] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:13 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:14.114615 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3faaf5c7-f83b-45b1-b88e-30899a6f3a6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:14.115221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3faaf5c7-f83b-45b1-b88e-30899a6f3a6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:14.121708 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3faaf5c7-f83b-45b1-b88e-30899a6f3a6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:14.122188 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3faaf5c7-f83b-45b1-b88e-30899a6f3a6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:14.122657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3faaf5c7-f83b-45b1-b88e-30899a6f3a6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:14.123341 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3faaf5c7-f83b-45b1-b88e-30899a6f3a6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:14.123817 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3faaf5c7-f83b-45b1-b88e-30899a6f3a6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:14.124256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3faaf5c7-f83b-45b1-b88e-30899a6f3a6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:14.251924 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3faaf5c7-f83b-45b1-b88e-30899a6f3a6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:14.260383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3faaf5c7-f83b-45b1-b88e-30899a6f3a6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1723 microversion: 2.1 time: 0.149694 Oct 20 09:04:14.260939 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 271/548] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:14 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1723 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:14.976266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-cbcfca45-0caf-47b9-961f-77da90edbb5d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:14.976838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-cbcfca45-0caf-47b9-961f-77da90edbb5d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:14.983370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbcfca45-0caf-47b9-961f-77da90edbb5d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:14.983853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbcfca45-0caf-47b9-961f-77da90edbb5d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:14.984374 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbcfca45-0caf-47b9-961f-77da90edbb5d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:14.985067 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbcfca45-0caf-47b9-961f-77da90edbb5d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:14.985522 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbcfca45-0caf-47b9-961f-77da90edbb5d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:14.985989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbcfca45-0caf-47b9-961f-77da90edbb5d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:15.116867 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-cbcfca45-0caf-47b9-961f-77da90edbb5d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:15.126825 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-368930ae-9922-41fd-b7f2-793d720638e8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:15.127404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-cbcfca45-0caf-47b9-961f-77da90edbb5d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1681 microversion: 2.1 time: 0.154797 Oct 20 09:04:15.128042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 278/549] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:14 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1681 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:15.128485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-368930ae-9922-41fd-b7f2-793d720638e8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:15.136636 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-368930ae-9922-41fd-b7f2-793d720638e8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:15.137380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-368930ae-9922-41fd-b7f2-793d720638e8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:15.137819 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-368930ae-9922-41fd-b7f2-793d720638e8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:15.138575 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-368930ae-9922-41fd-b7f2-793d720638e8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:15.139087 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-368930ae-9922-41fd-b7f2-793d720638e8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:15.139544 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-368930ae-9922-41fd-b7f2-793d720638e8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:15.159161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:15.283668 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c2f07d74-4408-4ecd-b335-5f296debac7b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:15.284268 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c2f07d74-4408-4ecd-b335-5f296debac7b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:15.291055 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c2f07d74-4408-4ecd-b335-5f296debac7b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:15.291543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c2f07d74-4408-4ecd-b335-5f296debac7b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:15.291971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c2f07d74-4408-4ecd-b335-5f296debac7b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:15.292898 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c2f07d74-4408-4ecd-b335-5f296debac7b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:15.293426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c2f07d74-4408-4ecd-b335-5f296debac7b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:15.294040 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c2f07d74-4408-4ecd-b335-5f296debac7b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:15.299420 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:15.354732 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-368930ae-9922-41fd-b7f2-793d720638e8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:15.364046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-368930ae-9922-41fd-b7f2-793d720638e8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.242206 Oct 20 09:04:15.366321 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 272/550] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:15 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:15.446940 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c2f07d74-4408-4ecd-b335-5f296debac7b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:15.455568 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c2f07d74-4408-4ecd-b335-5f296debac7b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1700 microversion: 2.1 time: 0.175933 Oct 20 09:04:15.456692 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 279/551] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:15 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1700 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:15.486777 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2059fff1-4aa5-4bc9-9407-ec70f32877e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:15.492897 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2059fff1-4aa5-4bc9-9407-ec70f32877e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:15.493675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2059fff1-4aa5-4bc9-9407-ec70f32877e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:15.494750 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2059fff1-4aa5-4bc9-9407-ec70f32877e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:15.520512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2059fff1-4aa5-4bc9-9407-ec70f32877e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Going to try to start instance {{(pid=119068) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 20 09:04:15.526068 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-42bfc189-c318-43ee-9fff-c0f2cae7a026 req-4b4d39ca-3046-4f69-b54d-1d76053c49ec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aae7b08a-4c9a-4d12-9598-193afab4ca00", "name": "network-vif-deleted", "tag": "68cd5f68-1310-4334-b214-dc31475d3f8f"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:15.550829 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-42bfc189-c318-43ee-9fff-c0f2cae7a026 req-4b4d39ca-3046-4f69-b54d-1d76053c49ec service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:15.551250 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-42bfc189-c318-43ee-9fff-c0f2cae7a026 req-4b4d39ca-3046-4f69-b54d-1d76053c49ec service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:15.551678 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-42bfc189-c318-43ee-9fff-c0f2cae7a026 req-4b4d39ca-3046-4f69-b54d-1d76053c49ec service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:15.569201 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-42bfc189-c318-43ee-9fff-c0f2cae7a026 req-4b4d39ca-3046-4f69-b54d-1d76053c49ec service nova] Creating event network-vif-deleted:68cd5f68-1310-4334-b214-dc31475d3f8f for instance aae7b08a-4c9a-4d12-9598-193afab4ca00 on hv-cinder-maste Oct 20 09:04:15.574884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-42bfc189-c318-43ee-9fff-c0f2cae7a026 req-4b4d39ca-3046-4f69-b54d-1d76053c49ec service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052274 Oct 20 09:04:15.575893 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 280/552] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:04:15.581201 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2059fff1-4aa5-4bc9-9407-ec70f32877e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/action" status: 202 len: 0 microversion: 2.1 time: 0.104148 Oct 20 09:04:15.582246 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 273/553] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:04:15 2022] POST /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/action => generated 0 bytes in 106 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:04:15.590685 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b265a9e8-07bc-480e-a437-1988c8810dd3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:15.591532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b265a9e8-07bc-480e-a437-1988c8810dd3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:15.599035 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b265a9e8-07bc-480e-a437-1988c8810dd3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:15.599827 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b265a9e8-07bc-480e-a437-1988c8810dd3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:15.600352 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b265a9e8-07bc-480e-a437-1988c8810dd3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:15.601259 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b265a9e8-07bc-480e-a437-1988c8810dd3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:15.601800 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b265a9e8-07bc-480e-a437-1988c8810dd3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:15.602321 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b265a9e8-07bc-480e-a437-1988c8810dd3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:15.739159 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b265a9e8-07bc-480e-a437-1988c8810dd3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:15.748913 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b265a9e8-07bc-480e-a437-1988c8810dd3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1709 microversion: 2.1 time: 0.162565 Oct 20 09:04:15.749481 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 281/554] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:15 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1709 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:16.153278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fc21561d-701f-4c68-a9b4-971a53182ad2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:16.154641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-fc21561d-701f-4c68-a9b4-971a53182ad2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:16.162311 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc21561d-701f-4c68-a9b4-971a53182ad2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:16.163333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc21561d-701f-4c68-a9b4-971a53182ad2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:16.164163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc21561d-701f-4c68-a9b4-971a53182ad2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:16.164800 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc21561d-701f-4c68-a9b4-971a53182ad2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:16.165417 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc21561d-701f-4c68-a9b4-971a53182ad2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:16.166660 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc21561d-701f-4c68-a9b4-971a53182ad2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:16.234170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-fc21561d-701f-4c68-a9b4-971a53182ad2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:16.242043 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fc21561d-701f-4c68-a9b4-971a53182ad2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1325 microversion: 2.1 time: 0.093480 Oct 20 09:04:16.242738 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 274/555] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:16 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1325 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:16.392956 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9f9fa766-5aa8-442c-a108-763a9d0f9854 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:16.393561 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9f9fa766-5aa8-442c-a108-763a9d0f9854 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:16.399690 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f9fa766-5aa8-442c-a108-763a9d0f9854 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:16.400791 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f9fa766-5aa8-442c-a108-763a9d0f9854 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:16.401265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f9fa766-5aa8-442c-a108-763a9d0f9854 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:16.401979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f9fa766-5aa8-442c-a108-763a9d0f9854 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:16.402433 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f9fa766-5aa8-442c-a108-763a9d0f9854 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:16.402888 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f9fa766-5aa8-442c-a108-763a9d0f9854 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:16.546388 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9f9fa766-5aa8-442c-a108-763a9d0f9854 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:16.554152 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9f9fa766-5aa8-442c-a108-763a9d0f9854 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.165113 Oct 20 09:04:16.555023 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 282/556] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:16 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:16.773201 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c9fb4bd7-7699-41c1-9185-e256a962b79e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:16.773975 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c9fb4bd7-7699-41c1-9185-e256a962b79e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:16.782498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9fb4bd7-7699-41c1-9185-e256a962b79e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:16.783098 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9fb4bd7-7699-41c1-9185-e256a962b79e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:16.783599 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9fb4bd7-7699-41c1-9185-e256a962b79e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:16.784477 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9fb4bd7-7699-41c1-9185-e256a962b79e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:16.785005 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9fb4bd7-7699-41c1-9185-e256a962b79e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:16.785486 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c9fb4bd7-7699-41c1-9185-e256a962b79e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:16.927665 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c9fb4bd7-7699-41c1-9185-e256a962b79e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:16.936091 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c9fb4bd7-7699-41c1-9185-e256a962b79e tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1709 microversion: 2.1 time: 0.167011 Oct 20 09:04:16.936591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 275/557] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:16 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1709 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:17.265427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:17.266333 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:17.273641 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:17.274266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:17.274867 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:17.275643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:17.276166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:17.276652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:17.324109 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:17.324590 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:17.325019 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:17.325469 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lazy-loading 'fault' on Instance uuid aae7b08a-4c9a-4d12-9598-193afab4ca00 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:04:17.377069 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:17.388887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-80b5a84f-17b0-44ef-9499-c43f88173f47 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 200 len: 1330 microversion: 2.1 time: 0.127285 Oct 20 09:04:17.389472 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 283/558] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:17 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 1330 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:17.577837 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fe1620e0-2825-4687-a220-2cc712a376a7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:17.578447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-fe1620e0-2825-4687-a220-2cc712a376a7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:17.586268 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fe1620e0-2825-4687-a220-2cc712a376a7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:17.586782 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fe1620e0-2825-4687-a220-2cc712a376a7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:17.587250 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fe1620e0-2825-4687-a220-2cc712a376a7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:17.587952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fe1620e0-2825-4687-a220-2cc712a376a7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:17.588402 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fe1620e0-2825-4687-a220-2cc712a376a7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:17.588824 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fe1620e0-2825-4687-a220-2cc712a376a7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:17.722046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-fe1620e0-2825-4687-a220-2cc712a376a7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:17.730157 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fe1620e0-2825-4687-a220-2cc712a376a7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.156095 Oct 20 09:04:17.730736 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 276/559] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:17 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:17.855925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-35864e90-2449-4ccb-84e3-fff18930d629 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:17.857087 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-35864e90-2449-4ccb-84e3-fff18930d629 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:17.863589 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35864e90-2449-4ccb-84e3-fff18930d629 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:17.864067 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35864e90-2449-4ccb-84e3-fff18930d629 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:17.864516 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35864e90-2449-4ccb-84e3-fff18930d629 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:17.865292 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35864e90-2449-4ccb-84e3-fff18930d629 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:17.865727 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35864e90-2449-4ccb-84e3-fff18930d629 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:17.866175 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-35864e90-2449-4ccb-84e3-fff18930d629 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:17.949722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-35864e90-2449-4ccb-84e3-fff18930d629 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lazy-loading 'services' on Instance uuid 98592479-f7da-4df7-ad14-c6aea41d81e0 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:04:17.960408 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-64e44aec-1525-40c4-8827-3e6f299a5c82 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:17.961097 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-64e44aec-1525-40c4-8827-3e6f299a5c82 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:17.967517 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64e44aec-1525-40c4-8827-3e6f299a5c82 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:17.968000 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64e44aec-1525-40c4-8827-3e6f299a5c82 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:17.972488 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64e44aec-1525-40c4-8827-3e6f299a5c82 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:17.973252 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64e44aec-1525-40c4-8827-3e6f299a5c82 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:17.973885 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64e44aec-1525-40c4-8827-3e6f299a5c82 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:17.974315 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-64e44aec-1525-40c4-8827-3e6f299a5c82 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:18.029280 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-35864e90-2449-4ccb-84e3-fff18930d629 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "DELETE /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0/os-volume_attachments/18201c29-7bb8-495f-9606-f7fec1347fd1" status: 202 len: 0 microversion: 2.1 time: 0.177437 Oct 20 09:04:18.030511 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 284/560] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:04:17 2022] DELETE /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0/os-volume_attachments/18201c29-7bb8-495f-9606-f7fec1347fd1 => generated 0 bytes in 179 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:04:18.118729 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-64e44aec-1525-40c4-8827-3e6f299a5c82 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:18.124845 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-64e44aec-1525-40c4-8827-3e6f299a5c82 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1709 microversion: 2.1 time: 0.168176 Oct 20 09:04:18.129929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 277/561] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:17 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1709 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:18.214725 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-93eacbd5-b805-448c-bbb2-b2d2f77fa6e7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:18.217352 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-93eacbd5-b805-448c-bbb2-b2d2f77fa6e7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:18.223581 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93eacbd5-b805-448c-bbb2-b2d2f77fa6e7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:18.223992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93eacbd5-b805-448c-bbb2-b2d2f77fa6e7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:18.224413 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93eacbd5-b805-448c-bbb2-b2d2f77fa6e7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:18.225109 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93eacbd5-b805-448c-bbb2-b2d2f77fa6e7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:18.225542 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93eacbd5-b805-448c-bbb2-b2d2f77fa6e7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:18.225955 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93eacbd5-b805-448c-bbb2-b2d2f77fa6e7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:18.250032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-93eacbd5-b805-448c-bbb2-b2d2f77fa6e7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lazy-loading 'flavor' on Instance uuid ea0966d3-1311-4690-a69f-d3df935a76e1 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:04:18.412817 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:18.413465 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:18.420235 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:18.420747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:18.421161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:18.421853 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:18.422302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:18.422743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:18.437771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 HTTP/1.1 Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:04:18.439469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:04:18.441142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': 'aae7b08a-4c9a-4d12-9598-193afab4ca00'} Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:04:18.441967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:04:18.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:18.444414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:04:18.444414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-9981e874-4edc-4510-8d1d-7a8b2827426d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 404 len: 111 microversion: 2.1 time: 0.032291 Oct 20 09:04:18.444414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 278/562] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:18 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 20 09:04:18.449414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1fd8cd3b-08b3-4ad2-8b65-f6ce0359593c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '.version_select at 0x7f5afd24c790>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:18.471809 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1fd8cd3b-08b3-4ad2-8b65-f6ce0359593c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-528204325" status: 202 len: 0 microversion: 2.1 time: 0.025944 Oct 20 09:04:18.472373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 279/563] 10.222.0.36 () {58 vars in 1218 bytes} [Thu Oct 20 09:04:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-528204325 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 20 09:04:18.533775 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-93eacbd5-b805-448c-bbb2-b2d2f77fa6e7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/action" status: 200 len: 32 microversion: 2.1 time: 0.323922 Oct 20 09:04:18.534860 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 285/564] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:04:18 2022] POST /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/action => generated 32 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 20 09:04:18.544293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d4dedcca-091b-4e3d-9636-db67e45480ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:18.544994 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d4dedcca-091b-4e3d-9636-db67e45480ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:18.552003 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d4dedcca-091b-4e3d-9636-db67e45480ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:18.552495 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d4dedcca-091b-4e3d-9636-db67e45480ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:18.552936 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d4dedcca-091b-4e3d-9636-db67e45480ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:18.553688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d4dedcca-091b-4e3d-9636-db67e45480ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:18.554145 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d4dedcca-091b-4e3d-9636-db67e45480ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:18.554603 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d4dedcca-091b-4e3d-9636-db67e45480ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:18.699549 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d4dedcca-091b-4e3d-9636-db67e45480ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:18.707766 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d4dedcca-091b-4e3d-9636-db67e45480ce tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.167671 Oct 20 09:04:18.708322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 280/565] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:18 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:18.753917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d1cd32c1-db9d-4799-9b06-0a99aa37b444 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:18.754544 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d1cd32c1-db9d-4799-9b06-0a99aa37b444 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:18.761127 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cd32c1-db9d-4799-9b06-0a99aa37b444 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:18.761602 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cd32c1-db9d-4799-9b06-0a99aa37b444 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:18.762028 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cd32c1-db9d-4799-9b06-0a99aa37b444 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:18.762746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cd32c1-db9d-4799-9b06-0a99aa37b444 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:18.763354 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cd32c1-db9d-4799-9b06-0a99aa37b444 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:18.763774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cd32c1-db9d-4799-9b06-0a99aa37b444 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:18.998966 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d1cd32c1-db9d-4799-9b06-0a99aa37b444 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:19.006753 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d1cd32c1-db9d-4799-9b06-0a99aa37b444 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.256997 Oct 20 09:04:19.007254 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 286/566] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:18 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:19.148220 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-380e0121-8846-4f92-821b-9c919df1b9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:19.149196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-380e0121-8846-4f92-821b-9c919df1b9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:19.157173 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-380e0121-8846-4f92-821b-9c919df1b9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:19.164441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-380e0121-8846-4f92-821b-9c919df1b9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:19.165283 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-380e0121-8846-4f92-821b-9c919df1b9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:19.172031 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-380e0121-8846-4f92-821b-9c919df1b9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:19.180566 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-380e0121-8846-4f92-821b-9c919df1b9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:19.181195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-380e0121-8846-4f92-821b-9c919df1b9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:19.260622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-93dc5537-62ad-42e5-9c54-1dc7136256c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:19.266932 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-93dc5537-62ad-42e5-9c54-1dc7136256c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:19.284549 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93dc5537-62ad-42e5-9c54-1dc7136256c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:19.285029 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93dc5537-62ad-42e5-9c54-1dc7136256c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:19.285462 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93dc5537-62ad-42e5-9c54-1dc7136256c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:19.286782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93dc5537-62ad-42e5-9c54-1dc7136256c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:19.287268 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93dc5537-62ad-42e5-9c54-1dc7136256c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:19.287689 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93dc5537-62ad-42e5-9c54-1dc7136256c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:19.323711 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-380e0121-8846-4f92-821b-9c919df1b9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:19.332161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-380e0121-8846-4f92-821b-9c919df1b9b5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1713 microversion: 2.1 time: 0.187759 Oct 20 09:04:19.332836 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 281/567] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:19 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1713 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:19.371695 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-93dc5537-62ad-42e5-9c54-1dc7136256c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lazy-loading 'services' on Instance uuid ec9ae57d-3e5a-490f-beba-d071e86a7b31 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:04:19.436584 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-74a0adcb-b14c-4745-a02c-eb79908f37da req-4aa08b4c-85c5-459e-950d-721304f5762b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b1d0b97-f53e-40dc-895f-48be8de838ee", "name": "network-vif-plugged", "status": "completed", "tag": "ea6a461f-d2ae-49cf-a87f-19296d8271f9"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:19.445643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-93dc5537-62ad-42e5-9c54-1dc7136256c5 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments/73bd08b5-9c2e-40b7-92db-80f1886711b9" status: 202 len: 0 microversion: 2.1 time: 0.189441 Oct 20 09:04:19.446350 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-74a0adcb-b14c-4745-a02c-eb79908f37da req-4aa08b4c-85c5-459e-950d-721304f5762b service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:19.447333 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 287/568] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:04:19 2022] DELETE /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments/73bd08b5-9c2e-40b7-92db-80f1886711b9 => generated 0 bytes in 192 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:04:19.448091 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-74a0adcb-b14c-4745-a02c-eb79908f37da req-4aa08b4c-85c5-459e-950d-721304f5762b service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:19.448588 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-74a0adcb-b14c-4745-a02c-eb79908f37da req-4aa08b4c-85c5-459e-950d-721304f5762b service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:19.455915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ae09c882-1df2-4d04-afa0-d12ddc8e8461 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:19.458388 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ae09c882-1df2-4d04-afa0-d12ddc8e8461 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:19.460564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-74a0adcb-b14c-4745-a02c-eb79908f37da req-4aa08b4c-85c5-459e-950d-721304f5762b service nova] Creating event network-vif-plugged:ea6a461f-d2ae-49cf-a87f-19296d8271f9 for instance 4b1d0b97-f53e-40dc-895f-48be8de838ee on hv-cinder-maste Oct 20 09:04:19.465763 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ae09c882-1df2-4d04-afa0-d12ddc8e8461 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:19.466263 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ae09c882-1df2-4d04-afa0-d12ddc8e8461 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:19.466740 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ae09c882-1df2-4d04-afa0-d12ddc8e8461 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:19.467434 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ae09c882-1df2-4d04-afa0-d12ddc8e8461 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:19.467883 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ae09c882-1df2-4d04-afa0-d12ddc8e8461 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:19.468129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-74a0adcb-b14c-4745-a02c-eb79908f37da req-4aa08b4c-85c5-459e-950d-721304f5762b service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035214 Oct 20 09:04:19.468304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ae09c882-1df2-4d04-afa0-d12ddc8e8461 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:19.469004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 282/569] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:04:19.497481 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ae09c882-1df2-4d04-afa0-d12ddc8e8461 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045848 Oct 20 09:04:19.498031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 288/570] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:04:19 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:04:19.733179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a85ae13c-9606-4898-8503-10d3f40978f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:19.733904 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a85ae13c-9606-4898-8503-10d3f40978f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:19.740031 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a85ae13c-9606-4898-8503-10d3f40978f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:19.740514 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a85ae13c-9606-4898-8503-10d3f40978f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:19.740951 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a85ae13c-9606-4898-8503-10d3f40978f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:19.741631 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a85ae13c-9606-4898-8503-10d3f40978f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:19.742089 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a85ae13c-9606-4898-8503-10d3f40978f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:19.742536 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a85ae13c-9606-4898-8503-10d3f40978f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:19.882037 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a85ae13c-9606-4898-8503-10d3f40978f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:19.891314 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a85ae13c-9606-4898-8503-10d3f40978f2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.162891 Oct 20 09:04:19.891929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 283/571] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:19 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:20.031658 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8695e971-8aec-484f-a12c-0e812a9cea2b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:20.032936 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8695e971-8aec-484f-a12c-0e812a9cea2b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:20.037090 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=81,nova_api:UPDATE=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:20.046884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8695e971-8aec-484f-a12c-0e812a9cea2b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:20.047494 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8695e971-8aec-484f-a12c-0e812a9cea2b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:20.049113 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8695e971-8aec-484f-a12c-0e812a9cea2b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:20.052563 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8695e971-8aec-484f-a12c-0e812a9cea2b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:20.053123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8695e971-8aec-484f-a12c-0e812a9cea2b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:20.053662 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8695e971-8aec-484f-a12c-0e812a9cea2b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:20.233860 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8695e971-8aec-484f-a12c-0e812a9cea2b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:20.241922 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8695e971-8aec-484f-a12c-0e812a9cea2b tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.216486 Oct 20 09:04:20.254807 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 289/572] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:20 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:20.510027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d15fc4e6-bad5-4e9f-a5db-f0e76c4fae1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:20.512141 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d15fc4e6-bad5-4e9f-a5db-f0e76c4fae1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:20.518959 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d15fc4e6-bad5-4e9f-a5db-f0e76c4fae1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:20.519443 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d15fc4e6-bad5-4e9f-a5db-f0e76c4fae1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:20.519886 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d15fc4e6-bad5-4e9f-a5db-f0e76c4fae1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:20.520596 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d15fc4e6-bad5-4e9f-a5db-f0e76c4fae1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:20.521048 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d15fc4e6-bad5-4e9f-a5db-f0e76c4fae1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:20.521481 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d15fc4e6-bad5-4e9f-a5db-f0e76c4fae1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:20.551603 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d15fc4e6-bad5-4e9f-a5db-f0e76c4fae1a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045807 Oct 20 09:04:20.552153 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 284/573] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:04:20 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:04:20.914613 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0b631dbf-0fde-4517-9af7-cc40433026af tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:20.915227 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0b631dbf-0fde-4517-9af7-cc40433026af tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:20.921165 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b631dbf-0fde-4517-9af7-cc40433026af tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:20.921643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b631dbf-0fde-4517-9af7-cc40433026af tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:20.922075 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b631dbf-0fde-4517-9af7-cc40433026af tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:20.922781 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b631dbf-0fde-4517-9af7-cc40433026af tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:20.923233 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b631dbf-0fde-4517-9af7-cc40433026af tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:20.923653 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b631dbf-0fde-4517-9af7-cc40433026af tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:21.055453 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0b631dbf-0fde-4517-9af7-cc40433026af tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:21.064968 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0b631dbf-0fde-4517-9af7-cc40433026af tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.154090 Oct 20 09:04:21.065765 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 290/574] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:20 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:21.266874 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-49371b7a-7e01-47bf-b642-5b1a535c02d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:21.267373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-49371b7a-7e01-47bf-b642-5b1a535c02d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:21.273635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-49371b7a-7e01-47bf-b642-5b1a535c02d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:21.274113 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-49371b7a-7e01-47bf-b642-5b1a535c02d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:21.274570 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-49371b7a-7e01-47bf-b642-5b1a535c02d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:21.275256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-49371b7a-7e01-47bf-b642-5b1a535c02d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:21.275705 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-49371b7a-7e01-47bf-b642-5b1a535c02d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:21.276121 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-49371b7a-7e01-47bf-b642-5b1a535c02d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:21.317488 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fa2e2155-e7c5-427a-b8f8-3369e2dc737c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:21.317984 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fa2e2155-e7c5-427a-b8f8-3369e2dc737c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:21.324553 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fa2e2155-e7c5-427a-b8f8-3369e2dc737c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:21.325037 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fa2e2155-e7c5-427a-b8f8-3369e2dc737c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:21.325469 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fa2e2155-e7c5-427a-b8f8-3369e2dc737c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:21.326160 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fa2e2155-e7c5-427a-b8f8-3369e2dc737c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:21.326636 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fa2e2155-e7c5-427a-b8f8-3369e2dc737c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:21.327056 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fa2e2155-e7c5-427a-b8f8-3369e2dc737c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:21.351965 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fa2e2155-e7c5-427a-b8f8-3369e2dc737c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:04:21.392041 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:21.413847 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-49371b7a-7e01-47bf-b642-5b1a535c02d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:21.422312 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-49371b7a-7e01-47bf-b642-5b1a535c02d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.159623 Oct 20 09:04:21.422918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 285/575] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:21 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:21.437604 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-fa2e2155-e7c5-427a-b8f8-3369e2dc737c tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "DELETE /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 204 len: 0 microversion: 2.1 time: 0.123649 Oct 20 09:04:21.438398 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 291/576] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:04:21 2022] DELETE /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 20 09:04:21.446254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-851d51bf-a41f-4282-a551-c9d920d71f46 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:21.446890 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-851d51bf-a41f-4282-a551-c9d920d71f46 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:21.452793 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-851d51bf-a41f-4282-a551-c9d920d71f46 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:21.453275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-851d51bf-a41f-4282-a551-c9d920d71f46 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:21.453713 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-851d51bf-a41f-4282-a551-c9d920d71f46 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:21.454397 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-851d51bf-a41f-4282-a551-c9d920d71f46 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:21.454894 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-851d51bf-a41f-4282-a551-c9d920d71f46 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:21.455322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-851d51bf-a41f-4282-a551-c9d920d71f46 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:21.565901 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d9dec836-437b-4708-b84c-30955e50b214 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:21.568424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d9dec836-437b-4708-b84c-30955e50b214 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:21.574771 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9dec836-437b-4708-b84c-30955e50b214 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:21.575256 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9dec836-437b-4708-b84c-30955e50b214 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:21.575704 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9dec836-437b-4708-b84c-30955e50b214 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:21.576691 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9dec836-437b-4708-b84c-30955e50b214 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:21.577152 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9dec836-437b-4708-b84c-30955e50b214 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:21.577577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9dec836-437b-4708-b84c-30955e50b214 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:21.588482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-851d51bf-a41f-4282-a551-c9d920d71f46 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:21.596746 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-851d51bf-a41f-4282-a551-c9d920d71f46 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1513 microversion: 2.1 time: 0.154174 Oct 20 09:04:21.597629 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 286/577] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:21 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1513 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:21.609207 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d9dec836-437b-4708-b84c-30955e50b214 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048620 Oct 20 09:04:21.609776 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 292/578] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:04:21 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:04:22.089581 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fed495f8-5ed8-4eb3-a00f-83b556ea37c7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:22.090439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-fed495f8-5ed8-4eb3-a00f-83b556ea37c7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:22.096519 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fed495f8-5ed8-4eb3-a00f-83b556ea37c7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:22.097007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fed495f8-5ed8-4eb3-a00f-83b556ea37c7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:22.097448 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fed495f8-5ed8-4eb3-a00f-83b556ea37c7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:22.098132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fed495f8-5ed8-4eb3-a00f-83b556ea37c7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:22.098614 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fed495f8-5ed8-4eb3-a00f-83b556ea37c7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:22.099041 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fed495f8-5ed8-4eb3-a00f-83b556ea37c7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:22.226153 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-fed495f8-5ed8-4eb3-a00f-83b556ea37c7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:22.234229 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fed495f8-5ed8-4eb3-a00f-83b556ea37c7 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.149037 Oct 20 09:04:22.234825 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 287/579] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:22 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:22.445179 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-adf4d75b-e9f8-41a7-b32a-343a4b0a4a54 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:22.445786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-adf4d75b-e9f8-41a7-b32a-343a4b0a4a54 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:22.452228 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-adf4d75b-e9f8-41a7-b32a-343a4b0a4a54 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:22.452708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-adf4d75b-e9f8-41a7-b32a-343a4b0a4a54 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:22.453142 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-adf4d75b-e9f8-41a7-b32a-343a4b0a4a54 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:22.453818 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-adf4d75b-e9f8-41a7-b32a-343a4b0a4a54 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:22.454267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-adf4d75b-e9f8-41a7-b32a-343a4b0a4a54 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:22.454726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-adf4d75b-e9f8-41a7-b32a-343a4b0a4a54 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:22.584463 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-adf4d75b-e9f8-41a7-b32a-343a4b0a4a54 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:22.592905 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-adf4d75b-e9f8-41a7-b32a-343a4b0a4a54 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1267 microversion: 2.1 time: 0.151546 Oct 20 09:04:22.593446 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 293/580] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:22 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:22.619760 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-aa0882c6-db82-4bdb-ad19-80afc9fed4d0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:22.620389 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-aa0882c6-db82-4bdb-ad19-80afc9fed4d0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:22.622357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e8e9845f-1aa6-41a9-a139-983a2ae13d81 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:22.624679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-e8e9845f-1aa6-41a9-a139-983a2ae13d81 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:22.626786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa0882c6-db82-4bdb-ad19-80afc9fed4d0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:22.627263 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa0882c6-db82-4bdb-ad19-80afc9fed4d0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:22.627693 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa0882c6-db82-4bdb-ad19-80afc9fed4d0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:22.628367 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa0882c6-db82-4bdb-ad19-80afc9fed4d0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:22.628808 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa0882c6-db82-4bdb-ad19-80afc9fed4d0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:22.629224 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-aa0882c6-db82-4bdb-ad19-80afc9fed4d0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:22.630985 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e8e9845f-1aa6-41a9-a139-983a2ae13d81 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:22.631494 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e8e9845f-1aa6-41a9-a139-983a2ae13d81 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:22.631935 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e8e9845f-1aa6-41a9-a139-983a2ae13d81 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:22.632621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e8e9845f-1aa6-41a9-a139-983a2ae13d81 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:22.633095 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e8e9845f-1aa6-41a9-a139-983a2ae13d81 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:22.633500 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e8e9845f-1aa6-41a9-a139-983a2ae13d81 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:22.662879 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e8e9845f-1aa6-41a9-a139-983a2ae13d81 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044304 Oct 20 09:04:22.663657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 294/581] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:04:22 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:04:22.757366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-aa0882c6-db82-4bdb-ad19-80afc9fed4d0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:22.765017 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-aa0882c6-db82-4bdb-ad19-80afc9fed4d0 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1513 microversion: 2.1 time: 0.149189 Oct 20 09:04:22.765582 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 288/582] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:22 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1513 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:23.257102 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3a234fa0-be0b-46f3-a65e-ea7bb77ddcd9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:23.257746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3a234fa0-be0b-46f3-a65e-ea7bb77ddcd9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:23.263827 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a234fa0-be0b-46f3-a65e-ea7bb77ddcd9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:23.264305 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a234fa0-be0b-46f3-a65e-ea7bb77ddcd9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:23.264816 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a234fa0-be0b-46f3-a65e-ea7bb77ddcd9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:23.265502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a234fa0-be0b-46f3-a65e-ea7bb77ddcd9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:23.265945 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a234fa0-be0b-46f3-a65e-ea7bb77ddcd9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:23.266435 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3a234fa0-be0b-46f3-a65e-ea7bb77ddcd9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:23.398652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3a234fa0-be0b-46f3-a65e-ea7bb77ddcd9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:23.409648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3a234fa0-be0b-46f3-a65e-ea7bb77ddcd9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.156108 Oct 20 09:04:23.410240 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 295/583] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:23 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:23.616553 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8360eff4-b1fb-482a-b0e0-11441bf9e483 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:23.617157 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8360eff4-b1fb-482a-b0e0-11441bf9e483 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:23.624114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8360eff4-b1fb-482a-b0e0-11441bf9e483 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:23.624663 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8360eff4-b1fb-482a-b0e0-11441bf9e483 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:23.625007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8360eff4-b1fb-482a-b0e0-11441bf9e483 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:23.625685 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8360eff4-b1fb-482a-b0e0-11441bf9e483 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:23.626135 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8360eff4-b1fb-482a-b0e0-11441bf9e483 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:23.626606 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8360eff4-b1fb-482a-b0e0-11441bf9e483 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:23.675957 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4431af88-9115-4b35-9185-eb9a8a4c5422 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:23.678227 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4431af88-9115-4b35-9185-eb9a8a4c5422 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:23.685486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4431af88-9115-4b35-9185-eb9a8a4c5422 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:23.685967 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4431af88-9115-4b35-9185-eb9a8a4c5422 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:23.686404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4431af88-9115-4b35-9185-eb9a8a4c5422 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:23.687146 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4431af88-9115-4b35-9185-eb9a8a4c5422 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:23.687593 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4431af88-9115-4b35-9185-eb9a8a4c5422 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:23.688014 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4431af88-9115-4b35-9185-eb9a8a4c5422 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:23.719556 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4431af88-9115-4b35-9185-eb9a8a4c5422 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047604 Oct 20 09:04:23.720130 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 296/584] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:04:23 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:04:23.754362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8360eff4-b1fb-482a-b0e0-11441bf9e483 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:23.762166 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8360eff4-b1fb-482a-b0e0-11441bf9e483 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1454 microversion: 2.1 time: 0.149636 Oct 20 09:04:23.762758 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 289/585] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:23 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1454 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:23.783816 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fc8de14a-4073-43dd-a32c-bfd0017bcf0e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:23.784416 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fc8de14a-4073-43dd-a32c-bfd0017bcf0e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:23.787993 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-11459fac-d348-446a-94e0-f2b37923e8a7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:23.788725 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-11459fac-d348-446a-94e0-f2b37923e8a7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:23.790437 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc8de14a-4073-43dd-a32c-bfd0017bcf0e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:23.791455 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc8de14a-4073-43dd-a32c-bfd0017bcf0e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:23.791885 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc8de14a-4073-43dd-a32c-bfd0017bcf0e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:23.792566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc8de14a-4073-43dd-a32c-bfd0017bcf0e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:23.793012 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc8de14a-4073-43dd-a32c-bfd0017bcf0e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:23.793465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc8de14a-4073-43dd-a32c-bfd0017bcf0e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:23.795083 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-11459fac-d348-446a-94e0-f2b37923e8a7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:23.795564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-11459fac-d348-446a-94e0-f2b37923e8a7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:23.796179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-11459fac-d348-446a-94e0-f2b37923e8a7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:23.796874 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-11459fac-d348-446a-94e0-f2b37923e8a7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:23.797332 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-11459fac-d348-446a-94e0-f2b37923e8a7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:23.797760 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-11459fac-d348-446a-94e0-f2b37923e8a7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:23.812818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:23.930845 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-11459fac-d348-446a-94e0-f2b37923e8a7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:23.933843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-fc8de14a-4073-43dd-a32c-bfd0017bcf0e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:23.939100 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-11459fac-d348-446a-94e0-f2b37923e8a7 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1513 microversion: 2.1 time: 0.154747 Oct 20 09:04:23.939666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 290/586] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:23 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1513 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:23.941688 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-fc8de14a-4073-43dd-a32c-bfd0017bcf0e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1454 microversion: 2.1 time: 0.161770 Oct 20 09:04:23.942284 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 297/587] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:23 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1454 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:24.433550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-94070faa-d0d3-41d9-b187-64712243fd55 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:24.434268 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-94070faa-d0d3-41d9-b187-64712243fd55 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:24.440373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94070faa-d0d3-41d9-b187-64712243fd55 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:24.440943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94070faa-d0d3-41d9-b187-64712243fd55 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:24.441456 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94070faa-d0d3-41d9-b187-64712243fd55 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:24.442220 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94070faa-d0d3-41d9-b187-64712243fd55 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:24.442771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94070faa-d0d3-41d9-b187-64712243fd55 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:24.443265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94070faa-d0d3-41d9-b187-64712243fd55 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:24.652794 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-94070faa-d0d3-41d9-b187-64712243fd55 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:24.663242 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-94070faa-d0d3-41d9-b187-64712243fd55 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.233625 Oct 20 09:04:24.675220 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 291/588] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:24 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:24.732505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-234e774f-e894-4a10-b9cc-63532ac16aef tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:24.735627 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-234e774f-e894-4a10-b9cc-63532ac16aef tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:24.742644 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-234e774f-e894-4a10-b9cc-63532ac16aef tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:24.743129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-234e774f-e894-4a10-b9cc-63532ac16aef tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:24.743569 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-234e774f-e894-4a10-b9cc-63532ac16aef tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:24.744445 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-234e774f-e894-4a10-b9cc-63532ac16aef tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:24.744928 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-234e774f-e894-4a10-b9cc-63532ac16aef tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:24.745318 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-234e774f-e894-4a10-b9cc-63532ac16aef tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:24.783355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-234e774f-e894-4a10-b9cc-63532ac16aef tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055581 Oct 20 09:04:24.789678 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 298/589] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:04:24 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31/os-volume_attachments => generated 25 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 20 09:04:24.810895 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-15b08548-ccdb-4fce-b593-59f2601070e4 req-a8d2747d-afff-4f15-8339-425086015461 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "98592479-f7da-4df7-ad14-c6aea41d81e0", "name": "network-vif-deleted", "tag": "eeeae5ec-ff15-4944-a558-e9f736d0ca1e"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:24.824725 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-15b08548-ccdb-4fce-b593-59f2601070e4 req-a8d2747d-afff-4f15-8339-425086015461 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:24.825344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-15b08548-ccdb-4fce-b593-59f2601070e4 req-a8d2747d-afff-4f15-8339-425086015461 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:24.826347 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-15b08548-ccdb-4fce-b593-59f2601070e4 req-a8d2747d-afff-4f15-8339-425086015461 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:24.841142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-15b08548-ccdb-4fce-b593-59f2601070e4 req-a8d2747d-afff-4f15-8339-425086015461 service nova] Creating event network-vif-deleted:eeeae5ec-ff15-4944-a558-e9f736d0ca1e for instance 98592479-f7da-4df7-ad14-c6aea41d81e0 on hv-cinder-maste Oct 20 09:04:24.846621 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-15b08548-ccdb-4fce-b593-59f2601070e4 req-a8d2747d-afff-4f15-8339-425086015461 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040075 Oct 20 09:04:24.847507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 292/590] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:04:24.860552 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ece2e2c3-bef0-4326-8648-d40438ad4ce2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:24.861046 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ece2e2c3-bef0-4326-8648-d40438ad4ce2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:24.867288 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ece2e2c3-bef0-4326-8648-d40438ad4ce2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:24.867774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ece2e2c3-bef0-4326-8648-d40438ad4ce2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:24.868207 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ece2e2c3-bef0-4326-8648-d40438ad4ce2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:24.869505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ece2e2c3-bef0-4326-8648-d40438ad4ce2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:24.869962 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ece2e2c3-bef0-4326-8648-d40438ad4ce2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:24.870387 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ece2e2c3-bef0-4326-8648-d40438ad4ce2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:24.897492 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ece2e2c3-bef0-4326-8648-d40438ad4ce2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:04:24.975881 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:24.979027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:24.985721 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ece2e2c3-bef0-4326-8648-d40438ad4ce2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 204 len: 0 microversion: 2.1 time: 0.128537 Oct 20 09:04:24.986500 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 299/591] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:04:24 2022] DELETE /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:04:24.994997 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:24.995925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:24.996547 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:24.997173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-64ac1fce-5f24-443a-875b-9ec37d6147d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:24.997939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-64ac1fce-5f24-443a-875b-9ec37d6147d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:24.998560 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:24.999178 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:24.999784 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:25.004399 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-64ac1fce-5f24-443a-875b-9ec37d6147d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:25.004935 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-64ac1fce-5f24-443a-875b-9ec37d6147d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:25.005535 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-64ac1fce-5f24-443a-875b-9ec37d6147d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:25.006513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-64ac1fce-5f24-443a-875b-9ec37d6147d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:25.006652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-64ac1fce-5f24-443a-875b-9ec37d6147d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:25.007093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-64ac1fce-5f24-443a-875b-9ec37d6147d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:25.040774 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:25.041402 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:25.041938 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:25.042524 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lazy-loading 'fault' on Instance uuid 98592479-f7da-4df7-ad14-c6aea41d81e0 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:04:25.085400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:25.093350 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bc159909-260b-406c-a558-3a7a2688b04d tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1309 microversion: 2.1 time: 0.134559 Oct 20 09:04:25.093918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 293/592] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:24 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1309 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:25.235294 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-64ac1fce-5f24-443a-875b-9ec37d6147d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:25.242980 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-64ac1fce-5f24-443a-875b-9ec37d6147d4 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1679 microversion: 2.1 time: 0.251406 Oct 20 09:04:25.243556 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 300/593] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:24 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1679 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:25.687928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-69c94b63-e13c-4947-b4dc-5dec1fb356a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:25.688555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-69c94b63-e13c-4947-b4dc-5dec1fb356a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:25.694906 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-69c94b63-e13c-4947-b4dc-5dec1fb356a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:25.695386 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-69c94b63-e13c-4947-b4dc-5dec1fb356a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:25.695819 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-69c94b63-e13c-4947-b4dc-5dec1fb356a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:25.696499 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-69c94b63-e13c-4947-b4dc-5dec1fb356a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:25.697098 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-69c94b63-e13c-4947-b4dc-5dec1fb356a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:25.697514 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-69c94b63-e13c-4947-b4dc-5dec1fb356a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:25.830364 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-69c94b63-e13c-4947-b4dc-5dec1fb356a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:25.839532 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-69c94b63-e13c-4947-b4dc-5dec1fb356a8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.155517 Oct 20 09:04:25.840112 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 294/594] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:25 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:25.986446 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-dca19783-f962-4153-9450-eb6167953124 req-995f99e6-a1a3-48d5-9b1e-71566bafa754 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4b1d0b97-f53e-40dc-895f-48be8de838ee", "tag": "ea6a461f-d2ae-49cf-a87f-19296d8271f9"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:25.996294 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dca19783-f962-4153-9450-eb6167953124 req-995f99e6-a1a3-48d5-9b1e-71566bafa754 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:25.996918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dca19783-f962-4153-9450-eb6167953124 req-995f99e6-a1a3-48d5-9b1e-71566bafa754 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:25.997350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dca19783-f962-4153-9450-eb6167953124 req-995f99e6-a1a3-48d5-9b1e-71566bafa754 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:26.008352 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-dca19783-f962-4153-9450-eb6167953124 req-995f99e6-a1a3-48d5-9b1e-71566bafa754 service nova] Creating event network-changed:ea6a461f-d2ae-49cf-a87f-19296d8271f9 for instance 4b1d0b97-f53e-40dc-895f-48be8de838ee on hv-cinder-maste Oct 20 09:04:26.013714 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-dca19783-f962-4153-9450-eb6167953124 req-995f99e6-a1a3-48d5-9b1e-71566bafa754 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031444 Oct 20 09:04:26.014303 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 301/595] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:26.115808 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:26.116420 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:26.122379 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:26.123009 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:26.123446 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:26.124129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:26.124577 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:26.124999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:26.156441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:26.156802 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:26.157338 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:26.157904 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lazy-loading 'fault' on Instance uuid 98592479-f7da-4df7-ad14-c6aea41d81e0 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:04:26.199956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:26.207628 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-9db5bd18-3f10-4fd4-a06c-0fa9abdcfc2f tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1309 microversion: 2.1 time: 0.095772 Oct 20 09:04:26.208180 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 295/596] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:26 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1309 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:26.266954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f5431fe6-bc01-4532-9949-163c2209d3c7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:26.267667 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f5431fe6-bc01-4532-9949-163c2209d3c7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:26.273648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5431fe6-bc01-4532-9949-163c2209d3c7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:26.274119 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5431fe6-bc01-4532-9949-163c2209d3c7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:26.274579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5431fe6-bc01-4532-9949-163c2209d3c7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:26.275768 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5431fe6-bc01-4532-9949-163c2209d3c7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:26.276208 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5431fe6-bc01-4532-9949-163c2209d3c7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:26.276626 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f5431fe6-bc01-4532-9949-163c2209d3c7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:26.410986 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f5431fe6-bc01-4532-9949-163c2209d3c7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:26.418684 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f5431fe6-bc01-4532-9949-163c2209d3c7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1679 microversion: 2.1 time: 0.155917 Oct 20 09:04:26.419252 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 302/597] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:26 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1679 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:26.864028 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a71c57b2-4da7-4ff7-92cf-bc01e8caef81 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:26.864563 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a71c57b2-4da7-4ff7-92cf-bc01e8caef81 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:26.871988 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a71c57b2-4da7-4ff7-92cf-bc01e8caef81 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:26.871988 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a71c57b2-4da7-4ff7-92cf-bc01e8caef81 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:26.872320 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a71c57b2-4da7-4ff7-92cf-bc01e8caef81 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:26.873192 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a71c57b2-4da7-4ff7-92cf-bc01e8caef81 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:26.873842 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a71c57b2-4da7-4ff7-92cf-bc01e8caef81 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:26.874808 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a71c57b2-4da7-4ff7-92cf-bc01e8caef81 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:27.080651 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a71c57b2-4da7-4ff7-92cf-bc01e8caef81 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:27.089855 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a71c57b2-4da7-4ff7-92cf-bc01e8caef81 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.229693 Oct 20 09:04:27.090724 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 296/598] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:26 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:27.231628 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:27.232242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:27.238643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:27.239126 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:27.239585 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:27.240282 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:27.240736 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:27.241159 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:27.271523 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:27.271980 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:27.272410 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:27.272849 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lazy-loading 'fault' on Instance uuid 98592479-f7da-4df7-ad14-c6aea41d81e0 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:04:27.326264 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7e0951446ad449d6b725fe6cc14df1f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9dc1a3b80bd34c8e8f691a9b47457f35', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:27.334289 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-89237cc4-334f-44dc-a4f6-7fbab63b1dc2 tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 200 len: 1309 microversion: 2.1 time: 0.107666 Oct 20 09:04:27.334896 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 303/599] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:27 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 1309 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:27.442179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c97140a4-ac05-4f1f-bd88-0f1c2bcca9a6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:27.443052 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c97140a4-ac05-4f1f-bd88-0f1c2bcca9a6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:27.449957 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c97140a4-ac05-4f1f-bd88-0f1c2bcca9a6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:27.451486 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c97140a4-ac05-4f1f-bd88-0f1c2bcca9a6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:27.452155 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c97140a4-ac05-4f1f-bd88-0f1c2bcca9a6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:27.453154 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c97140a4-ac05-4f1f-bd88-0f1c2bcca9a6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:27.453848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c97140a4-ac05-4f1f-bd88-0f1c2bcca9a6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:27.455203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c97140a4-ac05-4f1f-bd88-0f1c2bcca9a6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:27.478286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-9e8db39e-d5c2-406b-b8aa-eac93acdc539 req-81547964-8a2c-4a30-95b9-014a7852d449 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ec9ae57d-3e5a-490f-beba-d071e86a7b31", "name": "network-vif-unplugged", "status": "completed", "tag": "7681f5a2-5dbc-4513-b44a-9c70525ecfee"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:27.495256 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-9e8db39e-d5c2-406b-b8aa-eac93acdc539 req-81547964-8a2c-4a30-95b9-014a7852d449 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:27.496365 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-9e8db39e-d5c2-406b-b8aa-eac93acdc539 req-81547964-8a2c-4a30-95b9-014a7852d449 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:27.501802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-9e8db39e-d5c2-406b-b8aa-eac93acdc539 req-81547964-8a2c-4a30-95b9-014a7852d449 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:27.530407 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-9e8db39e-d5c2-406b-b8aa-eac93acdc539 req-81547964-8a2c-4a30-95b9-014a7852d449 service nova] Creating event network-vif-unplugged:7681f5a2-5dbc-4513-b44a-9c70525ecfee for instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 on hv-cinder-maste Oct 20 09:04:27.536068 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-9e8db39e-d5c2-406b-b8aa-eac93acdc539 req-81547964-8a2c-4a30-95b9-014a7852d449 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.062588 Oct 20 09:04:27.537580 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 304/600] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:27 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:27.713365 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c97140a4-ac05-4f1f-bd88-0f1c2bcca9a6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:27.721709 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c97140a4-ac05-4f1f-bd88-0f1c2bcca9a6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1679 microversion: 2.1 time: 0.283461 Oct 20 09:04:27.722600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 297/601] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:27 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1679 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:28.029729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-e73ca0d2-f716-45ed-8316-bc0b852d36a7 req-e1a25f3f-cc3f-477a-a902-8ea34b4d11cb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ec9ae57d-3e5a-490f-beba-d071e86a7b31", "name": "network-vif-deleted", "tag": "7681f5a2-5dbc-4513-b44a-9c70525ecfee"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:28.039596 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-e73ca0d2-f716-45ed-8316-bc0b852d36a7 req-e1a25f3f-cc3f-477a-a902-8ea34b4d11cb service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:28.040067 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-e73ca0d2-f716-45ed-8316-bc0b852d36a7 req-e1a25f3f-cc3f-477a-a902-8ea34b4d11cb service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:28.040519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-e73ca0d2-f716-45ed-8316-bc0b852d36a7 req-e1a25f3f-cc3f-477a-a902-8ea34b4d11cb service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:28.051838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-e73ca0d2-f716-45ed-8316-bc0b852d36a7 req-e1a25f3f-cc3f-477a-a902-8ea34b4d11cb service nova] Creating event network-vif-deleted:7681f5a2-5dbc-4513-b44a-9c70525ecfee for instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 on hv-cinder-maste Oct 20 09:04:28.058886 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-e73ca0d2-f716-45ed-8316-bc0b852d36a7 req-e1a25f3f-cc3f-477a-a902-8ea34b4d11cb service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033377 Oct 20 09:04:28.060268 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 305/602] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:28.113631 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e3dd06d7-de16-400c-ac17-d4f0b6925219 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:28.114334 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e3dd06d7-de16-400c-ac17-d4f0b6925219 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:28.121878 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3dd06d7-de16-400c-ac17-d4f0b6925219 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:28.122380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3dd06d7-de16-400c-ac17-d4f0b6925219 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:28.122849 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3dd06d7-de16-400c-ac17-d4f0b6925219 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:28.123558 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3dd06d7-de16-400c-ac17-d4f0b6925219 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:28.124022 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3dd06d7-de16-400c-ac17-d4f0b6925219 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:28.124443 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e3dd06d7-de16-400c-ac17-d4f0b6925219 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:28.292505 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e3dd06d7-de16-400c-ac17-d4f0b6925219 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:28.304313 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e3dd06d7-de16-400c-ac17-d4f0b6925219 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.194497 Oct 20 09:04:28.305355 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 298/603] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:28 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:28.358076 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:28.359311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] [instance: 98592479-f7da-4df7-ad14-c6aea41d81e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:28.366092 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:28.366605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:28.367033 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:28.367715 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:28.368336 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:28.369497 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:28.383215 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] HTTP exception thrown: Instance 98592479-f7da-4df7-ad14-c6aea41d81e0 could not be found. Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Request method failure captured: Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 HTTP/1.1 Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ6AAjqBCauEsn7-wVxFXyE5gkDP7ByGV9OJJcihX6NnQdtOCcweoduHZl0pTDp5mRnTkb3HVtP5FXtFisjEzcaT2-weJ4wxtTpLpMzepcqszoX2CJAOGbaor1x3yy44yWllx4hJ4EheGd-OtFLJGzyPVOSXc55J6lPkNDY43Dhs2cg Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 9dc1a3b80bd34c8e8f691a9b47457f35 Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1293092775 Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:04:28.384923 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/9dc1a3b80bd34c8e8f691a9b47457f35"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_9dc1a3b80bd34c8e8f691a9b47457f35", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/9dc1a3b80bd34c8e8f691a9b47457f35"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/9dc1a3b80bd34c8e8f691a9b47457f35"}]}] Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-VolumesSnapshotTestJSON-1293092775 Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 9dc1a3b80bd34c8e8f691a9b47457f35 Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1293092775 Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-VolumesSnapshotTestJSON-1293092775-project Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 7e0951446ad449d6b725fe6cc14df1f2 Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-VolumesSnapshotTestJSON-1293092775-project Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 98592479-f7da-4df7-ad14-c6aea41d81e0 could not be found. Oct 20 09:04:28.386436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '98592479-f7da-4df7-ad14-c6aea41d81e0'} Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 98592479-f7da-4df7-ad14-c6aea41d81e0 could not be found. Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:04:28.387266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 98592479-f7da-4df7-ad14-c6aea41d81e0 could not be found. Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 98592479-f7da-4df7-ad14-c6aea41d81e0 could not be found. Oct 20 09:04:28.388508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:04:28.389707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] Returning 404 to user: Instance 98592479-f7da-4df7-ad14-c6aea41d81e0 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:04:28.389707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9f85a0b9-8f4e-4bc8-9dd4-e7c5093eb9ae tempest-VolumesSnapshotTestJSON-1293092775 tempest-VolumesSnapshotTestJSON-1293092775-project] 10.222.0.36 "GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0" status: 404 len: 111 microversion: 2.1 time: 0.032386 Oct 20 09:04:28.389707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 306/604] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:28 2022] GET /compute/v2.1/servers/98592479-f7da-4df7-ad14-c6aea41d81e0 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:04:28.746455 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:28.747138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:28.753754 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:28.754254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:28.754721 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:28.755427 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:28.755883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:28.756311 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:28.797596 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:28.798650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:28.799254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:28.800044 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lazy-loading 'fault' on Instance uuid ec9ae57d-3e5a-490f-beba-d071e86a7b31 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:04:28.842562 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:28.849666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bd721355-f0ff-460b-adf2-4332e56269a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 200 len: 1330 microversion: 2.1 time: 0.107100 Oct 20 09:04:28.850245 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 299/605] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:28 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 1330 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:29.328366 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9d063619-2e63-42d8-b351-fb689d570d58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:29.328880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9d063619-2e63-42d8-b351-fb689d570d58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:29.334806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d063619-2e63-42d8-b351-fb689d570d58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:29.335280 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d063619-2e63-42d8-b351-fb689d570d58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:29.335708 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d063619-2e63-42d8-b351-fb689d570d58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:29.336386 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d063619-2e63-42d8-b351-fb689d570d58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:29.336825 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d063619-2e63-42d8-b351-fb689d570d58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:29.337745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d063619-2e63-42d8-b351-fb689d570d58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:29.464788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9d063619-2e63-42d8-b351-fb689d570d58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:29.473091 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9d063619-2e63-42d8-b351-fb689d570d58 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.148653 Oct 20 09:04:29.473647 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 307/606] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:29 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:29.874230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:29.874915 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:29.881196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:29.881678 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:29.882109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:29.882844 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:29.883672 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:29.884150 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:29.897745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 HTTP/1.1 Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:04:29.899699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:04:29.901240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': 'ec9ae57d-3e5a-490f-beba-d071e86a7b31'} Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:04:29.902064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:04:29.903509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:29.904799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:04:29.904799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e58d8cd8-e622-4d2e-be60-72b8e3044230 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 404 len: 111 microversion: 2.1 time: 0.031405 Oct 20 09:04:29.904799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 300/607] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:29 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:04:29.910136 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-af3ac192-9c13-4e09-ac1c-59f2fdb1c7d6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '.version_select at 0x7f5afbacde50>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:29.929992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-af3ac192-9c13-4e09-ac1c-59f2fdb1c7d6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-961266284" status: 202 len: 0 microversion: 2.1 time: 0.023491 Oct 20 09:04:29.930579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 308/608] 10.222.0.36 () {58 vars in 1218 bytes} [Thu Oct 20 09:04:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-961266284 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:04:30.497130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-409e3eae-697c-4788-a274-6f768b61e64b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:30.497737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-409e3eae-697c-4788-a274-6f768b61e64b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:30.500950 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=99,nova_api:DELETE=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:30.507558 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-409e3eae-697c-4788-a274-6f768b61e64b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:30.508055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-409e3eae-697c-4788-a274-6f768b61e64b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:30.508497 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-409e3eae-697c-4788-a274-6f768b61e64b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:30.509265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-409e3eae-697c-4788-a274-6f768b61e64b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:30.509718 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-409e3eae-697c-4788-a274-6f768b61e64b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:30.510144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-409e3eae-697c-4788-a274-6f768b61e64b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:30.534743 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d98591d9-18a5-4e69-adaa-ef6f7974ab75 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: .version_select at 0x7f5afb99a8b0>, body: {"keypair": {"name": "tempest-keypair-2061928950", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLD439yx8DaIG9nrTQkWVvKLw3mK6FtvhjyncZtRVVkwh9H1wC4XetFvp9uX3uzKEMCNeTGPfCMKwqFWv7haJNhsnCfiV+pARp2f6Gz76fLL6UtS4iZOoiqM/SMaHiP+tg=="}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:30.551414 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-d98591d9-18a5-4e69-adaa-ef6f7974ab75 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:04:30.555370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-d98591d9-18a5-4e69-adaa-ef6f7974ab75 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for user 8002ab26f6d64822aaa65aa730781b76 and project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:04:30.577746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d98591d9-18a5-4e69-adaa-ef6f7974ab75 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.047559 Oct 20 09:04:30.578337 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 309/609] 10.222.0.36 () {60 vars in 1159 bytes} [Thu Oct 20 09:04:30 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:04:30.644342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-409e3eae-697c-4788-a274-6f768b61e64b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:30.652759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-409e3eae-697c-4788-a274-6f768b61e64b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.160098 Oct 20 09:04:30.653522 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 301/610] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:30 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:31.677950 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d6240c31-63d0-401a-b427-d032aae96a4f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:31.678725 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d6240c31-63d0-401a-b427-d032aae96a4f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:31.684681 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d6240c31-63d0-401a-b427-d032aae96a4f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:31.685235 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d6240c31-63d0-401a-b427-d032aae96a4f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:31.685746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d6240c31-63d0-401a-b427-d032aae96a4f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:31.686703 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d6240c31-63d0-401a-b427-d032aae96a4f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:31.687120 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d6240c31-63d0-401a-b427-d032aae96a4f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:31.687605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d6240c31-63d0-401a-b427-d032aae96a4f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:31.705483 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:31.819744 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d6240c31-63d0-401a-b427-d032aae96a4f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:31.828338 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d6240c31-63d0-401a-b427-d032aae96a4f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.155026 Oct 20 09:04:31.828919 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 310/611] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:31 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:32.052288 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1848727524", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b8327be3-f028-4ebc-9b25-81152867d29e"}], "security_groups": [{"name": "tempest-securitygroup--2042255712"}], "key_name": "tempest-keypair-2061928950", "user_data": "}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:32.156223 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] validate_networks() for [('b8327be3-f028-4ebc-9b25-81152867d29e', None, None, None, None, None)] {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:04:32.526197 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:04:32.526665 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:04:32.527137 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:04:32.527542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:04:32.527951 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:04:32.536689 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:32.537163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:32.537601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:32.538316 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:32.538798 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:32.539222 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:32.555393 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:04:32.559953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for user 8002ab26f6d64822aaa65aa730781b76 and project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:04:32.570733 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:04:32.573603 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:04:32.619917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-af58f702-0f5a-4b26-8062-77c44ff7e1e6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.571656 Oct 20 09:04:32.620557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 302/612] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:04:32 2022] POST /compute/v2.1/servers => generated 402 bytes in 573 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:04:32.631399 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-dfd93da3-1b5d-42d5-8f2e-62a68179c5a8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:32.632004 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-dfd93da3-1b5d-42d5-8f2e-62a68179c5a8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:32.688619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-dfd93da3-1b5d-42d5-8f2e-62a68179c5a8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:32.695566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-dfd93da3-1b5d-42d5-8f2e-62a68179c5a8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1242 microversion: 2.1 time: 0.067912 Oct 20 09:04:32.696180 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 311/613] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:32 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1242 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:32.851122 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f5001903-c472-4f5e-97af-f22cbee5a3c9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:32.851731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f5001903-c472-4f5e-97af-f22cbee5a3c9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:32.860326 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f5001903-c472-4f5e-97af-f22cbee5a3c9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:32.860816 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f5001903-c472-4f5e-97af-f22cbee5a3c9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:32.861261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f5001903-c472-4f5e-97af-f22cbee5a3c9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:32.861978 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f5001903-c472-4f5e-97af-f22cbee5a3c9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:32.862431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f5001903-c472-4f5e-97af-f22cbee5a3c9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:32.862888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f5001903-c472-4f5e-97af-f22cbee5a3c9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:33.004973 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f5001903-c472-4f5e-97af-f22cbee5a3c9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:33.013123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f5001903-c472-4f5e-97af-f22cbee5a3c9 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.165900 Oct 20 09:04:33.013685 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 303/614] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:32 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:33.718294 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6ee3b60f-a83c-4ccf-b06e-b2ed073ab384 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:33.718924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6ee3b60f-a83c-4ccf-b06e-b2ed073ab384 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:33.725855 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ee3b60f-a83c-4ccf-b06e-b2ed073ab384 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:33.726330 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ee3b60f-a83c-4ccf-b06e-b2ed073ab384 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:33.726781 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ee3b60f-a83c-4ccf-b06e-b2ed073ab384 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:33.727454 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ee3b60f-a83c-4ccf-b06e-b2ed073ab384 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:33.727898 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ee3b60f-a83c-4ccf-b06e-b2ed073ab384 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:33.728817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ee3b60f-a83c-4ccf-b06e-b2ed073ab384 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:33.788501 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6ee3b60f-a83c-4ccf-b06e-b2ed073ab384 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:33.796676 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6ee3b60f-a83c-4ccf-b06e-b2ed073ab384 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1294 microversion: 2.1 time: 0.082539 Oct 20 09:04:33.797332 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 312/615] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:33 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1294 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:34.036624 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c2dbf9bd-06fe-4e88-9317-e19dc0286694 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:34.037232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c2dbf9bd-06fe-4e88-9317-e19dc0286694 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:34.044166 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2dbf9bd-06fe-4e88-9317-e19dc0286694 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:34.044650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2dbf9bd-06fe-4e88-9317-e19dc0286694 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:34.045086 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2dbf9bd-06fe-4e88-9317-e19dc0286694 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:34.046996 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2dbf9bd-06fe-4e88-9317-e19dc0286694 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:34.058130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2dbf9bd-06fe-4e88-9317-e19dc0286694 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:34.058404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2dbf9bd-06fe-4e88-9317-e19dc0286694 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:34.206748 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c2dbf9bd-06fe-4e88-9317-e19dc0286694 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:34.216495 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c2dbf9bd-06fe-4e88-9317-e19dc0286694 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.183614 Oct 20 09:04:34.217102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 304/616] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:34 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:34.819034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bfcf48f7-0a88-43ae-b43d-2e80af6f2915 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:34.819642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bfcf48f7-0a88-43ae-b43d-2e80af6f2915 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:34.827419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfcf48f7-0a88-43ae-b43d-2e80af6f2915 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:34.827915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfcf48f7-0a88-43ae-b43d-2e80af6f2915 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:34.828441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfcf48f7-0a88-43ae-b43d-2e80af6f2915 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:34.829154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfcf48f7-0a88-43ae-b43d-2e80af6f2915 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:34.829771 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfcf48f7-0a88-43ae-b43d-2e80af6f2915 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:34.830572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfcf48f7-0a88-43ae-b43d-2e80af6f2915 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:35.031200 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bfcf48f7-0a88-43ae-b43d-2e80af6f2915 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:35.038890 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bfcf48f7-0a88-43ae-b43d-2e80af6f2915 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1368 microversion: 2.1 time: 0.223700 Oct 20 09:04:35.039691 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 313/617] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:34 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1368 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:35.240460 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7a0e743f-9402-48c7-92b9-6e110d58be18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:35.241100 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7a0e743f-9402-48c7-92b9-6e110d58be18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:35.248205 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0e743f-9402-48c7-92b9-6e110d58be18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:35.248694 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0e743f-9402-48c7-92b9-6e110d58be18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:35.249490 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0e743f-9402-48c7-92b9-6e110d58be18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:35.250195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0e743f-9402-48c7-92b9-6e110d58be18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:35.250684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0e743f-9402-48c7-92b9-6e110d58be18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:35.251111 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0e743f-9402-48c7-92b9-6e110d58be18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:35.386867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7a0e743f-9402-48c7-92b9-6e110d58be18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:35.396185 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7a0e743f-9402-48c7-92b9-6e110d58be18 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.160639 Oct 20 09:04:35.396870 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 305/618] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:35 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:35.842343 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-49d1c132-0c0b-476d-bdb5-09107b0c905f req-7bc6074f-5a86-4d95-b28d-285c28af78b8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "06d8b443-0b7d-4287-8469-eb4a35863289", "tag": "db1c0050-0582-4d57-ba24-66fe0c293b1e"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:35.852535 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-49d1c132-0c0b-476d-bdb5-09107b0c905f req-7bc6074f-5a86-4d95-b28d-285c28af78b8 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:35.852738 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-49d1c132-0c0b-476d-bdb5-09107b0c905f req-7bc6074f-5a86-4d95-b28d-285c28af78b8 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:35.853110 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-49d1c132-0c0b-476d-bdb5-09107b0c905f req-7bc6074f-5a86-4d95-b28d-285c28af78b8 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:35.866239 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-49d1c132-0c0b-476d-bdb5-09107b0c905f req-7bc6074f-5a86-4d95-b28d-285c28af78b8 service nova] Creating event network-changed:db1c0050-0582-4d57-ba24-66fe0c293b1e for instance 06d8b443-0b7d-4287-8469-eb4a35863289 on hv-cinder-maste Oct 20 09:04:35.871538 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-49d1c132-0c0b-476d-bdb5-09107b0c905f req-7bc6074f-5a86-4d95-b28d-285c28af78b8 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033783 Oct 20 09:04:35.872144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 314/619] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:36.062138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-881d1f59-36ac-4259-810a-d7ab1d5c5528 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:36.063083 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-881d1f59-36ac-4259-810a-d7ab1d5c5528 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:36.069412 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-881d1f59-36ac-4259-810a-d7ab1d5c5528 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:36.069900 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-881d1f59-36ac-4259-810a-d7ab1d5c5528 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:36.070337 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-881d1f59-36ac-4259-810a-d7ab1d5c5528 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:36.071066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-881d1f59-36ac-4259-810a-d7ab1d5c5528 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:36.071516 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-881d1f59-36ac-4259-810a-d7ab1d5c5528 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:36.072998 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-881d1f59-36ac-4259-810a-d7ab1d5c5528 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:36.202458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-881d1f59-36ac-4259-810a-d7ab1d5c5528 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:36.210131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-881d1f59-36ac-4259-810a-d7ab1d5c5528 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1368 microversion: 2.1 time: 0.151912 Oct 20 09:04:36.210745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 306/620] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:36 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1368 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:36.421028 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6d3c5484-f74d-4b45-90d1-35d4d36d07df tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:36.421640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6d3c5484-f74d-4b45-90d1-35d4d36d07df tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:36.427798 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3c5484-f74d-4b45-90d1-35d4d36d07df tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:36.428278 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3c5484-f74d-4b45-90d1-35d4d36d07df tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:36.428716 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3c5484-f74d-4b45-90d1-35d4d36d07df tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:36.429410 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3c5484-f74d-4b45-90d1-35d4d36d07df tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:36.429865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3c5484-f74d-4b45-90d1-35d4d36d07df tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:36.430289 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3c5484-f74d-4b45-90d1-35d4d36d07df tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:36.557097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6d3c5484-f74d-4b45-90d1-35d4d36d07df tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:36.565271 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6d3c5484-f74d-4b45-90d1-35d4d36d07df tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.148643 Oct 20 09:04:36.565836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 315/621] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:36 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:37.233593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1c3a09ba-fab7-4bcb-b8bb-bb125340ba82 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:37.234132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1c3a09ba-fab7-4bcb-b8bb-bb125340ba82 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:37.241263 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c3a09ba-fab7-4bcb-b8bb-bb125340ba82 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:37.241743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c3a09ba-fab7-4bcb-b8bb-bb125340ba82 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:37.242173 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c3a09ba-fab7-4bcb-b8bb-bb125340ba82 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:37.242894 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c3a09ba-fab7-4bcb-b8bb-bb125340ba82 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:37.243341 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c3a09ba-fab7-4bcb-b8bb-bb125340ba82 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:37.244507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c3a09ba-fab7-4bcb-b8bb-bb125340ba82 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:37.375913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1c3a09ba-fab7-4bcb-b8bb-bb125340ba82 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:37.383617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1c3a09ba-fab7-4bcb-b8bb-bb125340ba82 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1368 microversion: 2.1 time: 0.153953 Oct 20 09:04:37.384190 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 307/622] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:37 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1368 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:37.588576 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6a8e60e6-28df-4540-8019-b51e49d5bca0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:37.589187 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6a8e60e6-28df-4540-8019-b51e49d5bca0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:37.595918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a8e60e6-28df-4540-8019-b51e49d5bca0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:37.596397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a8e60e6-28df-4540-8019-b51e49d5bca0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:37.596831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a8e60e6-28df-4540-8019-b51e49d5bca0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:37.597517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a8e60e6-28df-4540-8019-b51e49d5bca0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:37.597971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a8e60e6-28df-4540-8019-b51e49d5bca0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:37.598397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a8e60e6-28df-4540-8019-b51e49d5bca0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:37.751307 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6a8e60e6-28df-4540-8019-b51e49d5bca0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:37.760103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6a8e60e6-28df-4540-8019-b51e49d5bca0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.174716 Oct 20 09:04:37.760675 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 316/623] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:37 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:38.405094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d153e5a3-d656-491c-9e70-1188876b2711 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:38.405723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d153e5a3-d656-491c-9e70-1188876b2711 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:38.412155 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d153e5a3-d656-491c-9e70-1188876b2711 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:38.412627 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d153e5a3-d656-491c-9e70-1188876b2711 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:38.413054 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d153e5a3-d656-491c-9e70-1188876b2711 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:38.413736 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d153e5a3-d656-491c-9e70-1188876b2711 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:38.414179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d153e5a3-d656-491c-9e70-1188876b2711 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:38.414628 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d153e5a3-d656-491c-9e70-1188876b2711 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:38.418412 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:38.539318 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d153e5a3-d656-491c-9e70-1188876b2711 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:38.546727 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d153e5a3-d656-491c-9e70-1188876b2711 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1368 microversion: 2.1 time: 0.145477 Oct 20 09:04:38.547290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 308/624] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:38 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1368 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:38.784935 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-48d0cb45-707b-4a54-9d51-3588e40c896d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:38.785464 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-48d0cb45-707b-4a54-9d51-3588e40c896d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:38.792352 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48d0cb45-707b-4a54-9d51-3588e40c896d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:38.792831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48d0cb45-707b-4a54-9d51-3588e40c896d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:38.793265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48d0cb45-707b-4a54-9d51-3588e40c896d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:38.793952 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48d0cb45-707b-4a54-9d51-3588e40c896d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:38.794709 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48d0cb45-707b-4a54-9d51-3588e40c896d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:38.795133 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-48d0cb45-707b-4a54-9d51-3588e40c896d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:38.920786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-48d0cb45-707b-4a54-9d51-3588e40c896d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:38.928968 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-48d0cb45-707b-4a54-9d51-3588e40c896d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.148325 Oct 20 09:04:38.929537 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 317/625] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:38 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:39.568607 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4235acdc-6d65-434e-9c3d-184e4922a4b3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:39.569212 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4235acdc-6d65-434e-9c3d-184e4922a4b3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:39.576092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4235acdc-6d65-434e-9c3d-184e4922a4b3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:39.576572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4235acdc-6d65-434e-9c3d-184e4922a4b3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:39.577003 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4235acdc-6d65-434e-9c3d-184e4922a4b3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:39.577683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4235acdc-6d65-434e-9c3d-184e4922a4b3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:39.578129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4235acdc-6d65-434e-9c3d-184e4922a4b3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:39.578576 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4235acdc-6d65-434e-9c3d-184e4922a4b3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:39.706588 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4235acdc-6d65-434e-9c3d-184e4922a4b3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:39.714626 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4235acdc-6d65-434e-9c3d-184e4922a4b3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1368 microversion: 2.1 time: 0.149871 Oct 20 09:04:39.715439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 309/626] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:39 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1368 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:39.952486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6c6587ec-57c6-4985-87cf-580fa83d00ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:39.953006 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6c6587ec-57c6-4985-87cf-580fa83d00ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:39.959623 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c6587ec-57c6-4985-87cf-580fa83d00ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:39.960096 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c6587ec-57c6-4985-87cf-580fa83d00ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:39.960525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c6587ec-57c6-4985-87cf-580fa83d00ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:39.961201 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c6587ec-57c6-4985-87cf-580fa83d00ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:39.961642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c6587ec-57c6-4985-87cf-580fa83d00ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:39.962056 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6c6587ec-57c6-4985-87cf-580fa83d00ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:40.085655 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6c6587ec-57c6-4985-87cf-580fa83d00ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:40.094651 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6c6587ec-57c6-4985-87cf-580fa83d00ed tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.146026 Oct 20 09:04:40.095213 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 318/627] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:39 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:40.737022 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-25f84336-b11d-4dc1-90da-a41778c77c11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:40.737584 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-25f84336-b11d-4dc1-90da-a41778c77c11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:40.744400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-25f84336-b11d-4dc1-90da-a41778c77c11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:40.744890 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-25f84336-b11d-4dc1-90da-a41778c77c11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:40.745322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-25f84336-b11d-4dc1-90da-a41778c77c11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:40.745995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-25f84336-b11d-4dc1-90da-a41778c77c11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:40.746440 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-25f84336-b11d-4dc1-90da-a41778c77c11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:40.746881 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-25f84336-b11d-4dc1-90da-a41778c77c11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:40.878182 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-25f84336-b11d-4dc1-90da-a41778c77c11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:40.886328 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-25f84336-b11d-4dc1-90da-a41778c77c11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1368 microversion: 2.1 time: 0.153222 Oct 20 09:04:40.887085 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 310/628] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:40 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1368 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:41.117384 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bdd1a89b-b877-4287-8dd0-5ac505e61e43 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:41.117986 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bdd1a89b-b877-4287-8dd0-5ac505e61e43 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:41.124460 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdd1a89b-b877-4287-8dd0-5ac505e61e43 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:41.124962 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdd1a89b-b877-4287-8dd0-5ac505e61e43 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:41.125394 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdd1a89b-b877-4287-8dd0-5ac505e61e43 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:41.126069 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdd1a89b-b877-4287-8dd0-5ac505e61e43 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:41.126535 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdd1a89b-b877-4287-8dd0-5ac505e61e43 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:41.126955 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdd1a89b-b877-4287-8dd0-5ac505e61e43 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:41.264455 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bdd1a89b-b877-4287-8dd0-5ac505e61e43 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:41.273064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bdd1a89b-b877-4287-8dd0-5ac505e61e43 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.159653 Oct 20 09:04:41.273625 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 319/629] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:41 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:41.909222 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6655e878-79cd-4eec-8905-dbced3f714f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:41.909752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6655e878-79cd-4eec-8905-dbced3f714f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:41.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6655e878-79cd-4eec-8905-dbced3f714f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:41.917283 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6655e878-79cd-4eec-8905-dbced3f714f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:41.917714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6655e878-79cd-4eec-8905-dbced3f714f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:41.918402 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6655e878-79cd-4eec-8905-dbced3f714f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:41.918873 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6655e878-79cd-4eec-8905-dbced3f714f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:41.919290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6655e878-79cd-4eec-8905-dbced3f714f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:42.049005 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6655e878-79cd-4eec-8905-dbced3f714f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:42.057686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6655e878-79cd-4eec-8905-dbced3f714f2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1368 microversion: 2.1 time: 0.152391 Oct 20 09:04:42.058259 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 311/630] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:41 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1368 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:42.297151 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d57cdf11-ff9e-4d96-9ba6-c26435668098 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:42.297752 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d57cdf11-ff9e-4d96-9ba6-c26435668098 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:42.304106 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d57cdf11-ff9e-4d96-9ba6-c26435668098 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:42.304573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d57cdf11-ff9e-4d96-9ba6-c26435668098 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:42.304988 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d57cdf11-ff9e-4d96-9ba6-c26435668098 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:42.305661 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d57cdf11-ff9e-4d96-9ba6-c26435668098 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:42.306106 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d57cdf11-ff9e-4d96-9ba6-c26435668098 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:42.307077 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d57cdf11-ff9e-4d96-9ba6-c26435668098 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:42.446309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d57cdf11-ff9e-4d96-9ba6-c26435668098 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:42.454603 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d57cdf11-ff9e-4d96-9ba6-c26435668098 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.161389 Oct 20 09:04:42.455179 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 320/631] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:42 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:43.080021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7f84bed1-a0e8-4fbe-916f-706a9b7dd23e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:43.080637 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7f84bed1-a0e8-4fbe-916f-706a9b7dd23e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:43.082464 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:43.083149 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=6 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:43.089920 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7f84bed1-a0e8-4fbe-916f-706a9b7dd23e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:43.090694 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7f84bed1-a0e8-4fbe-916f-706a9b7dd23e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:43.091144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7f84bed1-a0e8-4fbe-916f-706a9b7dd23e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:43.091833 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7f84bed1-a0e8-4fbe-916f-706a9b7dd23e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:43.092287 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7f84bed1-a0e8-4fbe-916f-706a9b7dd23e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:43.092715 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7f84bed1-a0e8-4fbe-916f-706a9b7dd23e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:43.188071 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-ae05f65c-8621-4eb9-ba8c-dbe4d7c8a554 req-71dc3236-c585-4c17-a345-d7fe845ecab9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "06d8b443-0b7d-4287-8469-eb4a35863289", "name": "network-vif-plugged", "status": "completed", "tag": "db1c0050-0582-4d57-ba24-66fe0c293b1e"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:43.193559 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=6 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:43.199751 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-ae05f65c-8621-4eb9-ba8c-dbe4d7c8a554 req-71dc3236-c585-4c17-a345-d7fe845ecab9 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:43.200447 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-ae05f65c-8621-4eb9-ba8c-dbe4d7c8a554 req-71dc3236-c585-4c17-a345-d7fe845ecab9 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:43.201245 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-ae05f65c-8621-4eb9-ba8c-dbe4d7c8a554 req-71dc3236-c585-4c17-a345-d7fe845ecab9 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:43.213400 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-ae05f65c-8621-4eb9-ba8c-dbe4d7c8a554 req-71dc3236-c585-4c17-a345-d7fe845ecab9 service nova] Creating event network-vif-plugged:db1c0050-0582-4d57-ba24-66fe0c293b1e for instance 06d8b443-0b7d-4287-8469-eb4a35863289 on hv-cinder-maste Oct 20 09:04:43.218913 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-ae05f65c-8621-4eb9-ba8c-dbe4d7c8a554 req-71dc3236-c585-4c17-a345-d7fe845ecab9 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035435 Oct 20 09:04:43.219512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 321/632] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:43.227589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7f84bed1-a0e8-4fbe-916f-706a9b7dd23e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:43.236281 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7f84bed1-a0e8-4fbe-916f-706a9b7dd23e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1368 microversion: 2.1 time: 0.160115 Oct 20 09:04:43.236911 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 312/633] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:43 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1368 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:43.478104 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-799e93d7-df11-48f5-9593-21bad4f70001 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:43.479308 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-799e93d7-df11-48f5-9593-21bad4f70001 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:43.485300 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-799e93d7-df11-48f5-9593-21bad4f70001 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:43.485769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-799e93d7-df11-48f5-9593-21bad4f70001 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:43.486196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-799e93d7-df11-48f5-9593-21bad4f70001 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:43.486894 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-799e93d7-df11-48f5-9593-21bad4f70001 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:43.487338 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-799e93d7-df11-48f5-9593-21bad4f70001 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:43.487750 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-799e93d7-df11-48f5-9593-21bad4f70001 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:43.617062 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-799e93d7-df11-48f5-9593-21bad4f70001 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:43.626331 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-799e93d7-df11-48f5-9593-21bad4f70001 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1733 microversion: 2.1 time: 0.152865 Oct 20 09:04:43.626935 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 322/634] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:43 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1733 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:44.259762 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6456dfbc-6b47-4650-bccc-d0b05e95ee35 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:44.260414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6456dfbc-6b47-4650-bccc-d0b05e95ee35 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:44.267074 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6456dfbc-6b47-4650-bccc-d0b05e95ee35 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:44.267557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6456dfbc-6b47-4650-bccc-d0b05e95ee35 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:44.267993 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6456dfbc-6b47-4650-bccc-d0b05e95ee35 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:44.268677 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6456dfbc-6b47-4650-bccc-d0b05e95ee35 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:44.269131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6456dfbc-6b47-4650-bccc-d0b05e95ee35 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:44.269557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6456dfbc-6b47-4650-bccc-d0b05e95ee35 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:44.396218 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6456dfbc-6b47-4650-bccc-d0b05e95ee35 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:44.404318 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6456dfbc-6b47-4650-bccc-d0b05e95ee35 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1368 microversion: 2.1 time: 0.148518 Oct 20 09:04:44.404896 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 313/635] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:44 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1368 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:44.649017 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-562fa4b7-e7c0-483f-ae1a-076dc4434c07 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:44.649630 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-562fa4b7-e7c0-483f-ae1a-076dc4434c07 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:44.656844 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-562fa4b7-e7c0-483f-ae1a-076dc4434c07 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:44.657326 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-562fa4b7-e7c0-483f-ae1a-076dc4434c07 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:44.657762 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-562fa4b7-e7c0-483f-ae1a-076dc4434c07 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:44.658452 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-562fa4b7-e7c0-483f-ae1a-076dc4434c07 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:44.658932 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-562fa4b7-e7c0-483f-ae1a-076dc4434c07 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:44.659357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-562fa4b7-e7c0-483f-ae1a-076dc4434c07 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:44.785389 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-562fa4b7-e7c0-483f-ae1a-076dc4434c07 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:44.793528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-562fa4b7-e7c0-483f-ae1a-076dc4434c07 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1713 microversion: 2.1 time: 0.148415 Oct 20 09:04:44.794098 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 323/636] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:44 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1713 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:44.905147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:44.905646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:44.911579 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:44.912061 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:44.912495 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:44.914133 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:44.914133 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:44.914133 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:45.036428 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] HTTP exception thrown: Cannot 'detach_volume' instance ea0966d3-1311-4690-a69f-d3df935a76e1 while it is in vm_state rescued Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Request method failure captured: Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: DELETE /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments/408e1832-d211-409c-a48e-dd0ec66831c0 HTTP/1.1 Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ44IzvbrfISoSWQdaz7A4Y4mCB866TdY__L0_HKNgMuGtSn76PJ_HBHhaSlJVS78rXHSeDqK-8OZHSlR6XDrjto3KeVysuo0baUN07Q2QroH6vEp0-TETCDD7MgP6eUx6PZ5XfPA9gmR0Yq3sbUMXAjpaWW4IJsJYamevW56vxD4ZM Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: d24a1fc0e9344af181d49cfc58f0c88b Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:04:45.037941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/d24a1fc0e9344af181d49cfc58f0c88b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_d24a1fc0e9344af181d49cfc58f0c88b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/d24a1fc0e9344af181d49cfc58f0c88b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/d24a1fc0e9344af181d49cfc58f0c88b"}]}] Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: d24a1fc0e9344af181d49cfc58f0c88b Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-ServerRescueNegativeTestJSON-1468667757-project Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 7f932695a5994c5790d14c210ea65326 Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1468667757-project Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:04:45.039530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Cannot 'detach_volume' instance ea0966d3-1311-4690-a69f-d3df935a76e1 while it is in vm_state rescued Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'server_id': 'ea0966d3-1311-4690-a69f-d3df935a76e1', 'id': '408e1832-d211-409c-a48e-dd0ec66831c0'} Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance ea0966d3-1311-4690-a69f-d3df935a76e1 in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:04:45.040511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance ea0966d3-1311-4690-a69f-d3df935a76e1 while it is in vm_state rescued Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance ea0966d3-1311-4690-a69f-d3df935a76e1 while it is in vm_state rescued Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:04:45.042146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Returning 409 to user: Cannot 'detach_volume' instance ea0966d3-1311-4690-a69f-d3df935a76e1 while it is in vm_state rescued {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:04:45.043465 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c77986ca-f3bc-40b3-b090-be9cf8faa0f1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "DELETE /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments/408e1832-d211-409c-a48e-dd0ec66831c0" status: 409 len: 152 microversion: 2.1 time: 0.138783 Oct 20 09:04:45.043465 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 314/637] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:04:44 2022] DELETE /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments/408e1832-d211-409c-a48e-dd0ec66831c0 => generated 152 bytes in 140 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 20 09:04:45.048283 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b911aff7-0ce9-4d13-aa68-af99f8a21627 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:45.048815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b911aff7-0ce9-4d13-aa68-af99f8a21627 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:45.054687 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b911aff7-0ce9-4d13-aa68-af99f8a21627 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:45.055169 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b911aff7-0ce9-4d13-aa68-af99f8a21627 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:45.055603 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b911aff7-0ce9-4d13-aa68-af99f8a21627 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:45.056288 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b911aff7-0ce9-4d13-aa68-af99f8a21627 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:45.056743 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b911aff7-0ce9-4d13-aa68-af99f8a21627 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:45.057168 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b911aff7-0ce9-4d13-aa68-af99f8a21627 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:45.124641 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b911aff7-0ce9-4d13-aa68-af99f8a21627 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "POST /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/action" status: 202 len: 0 microversion: 2.1 time: 0.080035 Oct 20 09:04:45.125286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 324/638] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:04:45 2022] POST /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/action => generated 0 bytes in 81 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:04:45.133776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fdced0a7-9dfc-48ef-88e3-fc8b467736e2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:45.134407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-fdced0a7-9dfc-48ef-88e3-fc8b467736e2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:45.141671 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fdced0a7-9dfc-48ef-88e3-fc8b467736e2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:45.142179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fdced0a7-9dfc-48ef-88e3-fc8b467736e2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:45.142651 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fdced0a7-9dfc-48ef-88e3-fc8b467736e2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:45.143330 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fdced0a7-9dfc-48ef-88e3-fc8b467736e2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:45.143804 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fdced0a7-9dfc-48ef-88e3-fc8b467736e2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:45.144220 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fdced0a7-9dfc-48ef-88e3-fc8b467736e2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:45.274654 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-fdced0a7-9dfc-48ef-88e3-fc8b467736e2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:45.282921 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fdced0a7-9dfc-48ef-88e3-fc8b467736e2 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1721 microversion: 2.1 time: 0.152919 Oct 20 09:04:45.283475 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 315/639] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:45 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:45.431787 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a71dd204-c66d-421b-8c3a-652b1c408691 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:45.432500 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a71dd204-c66d-421b-8c3a-652b1c408691 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:45.435013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:45.440404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a71dd204-c66d-421b-8c3a-652b1c408691 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:45.440884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a71dd204-c66d-421b-8c3a-652b1c408691 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:45.441322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a71dd204-c66d-421b-8c3a-652b1c408691 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:45.442008 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a71dd204-c66d-421b-8c3a-652b1c408691 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:45.442485 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a71dd204-c66d-421b-8c3a-652b1c408691 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:45.442911 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a71dd204-c66d-421b-8c3a-652b1c408691 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:45.570573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a71dd204-c66d-421b-8c3a-652b1c408691 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:45.578807 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a71dd204-c66d-421b-8c3a-652b1c408691 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1368 microversion: 2.1 time: 0.151223 Oct 20 09:04:45.579368 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 325/640] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:45 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1368 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:46.306184 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bbdc6467-c69e-42eb-840b-36c3beb86fb5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:46.306734 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bbdc6467-c69e-42eb-840b-36c3beb86fb5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:46.313597 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbdc6467-c69e-42eb-840b-36c3beb86fb5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:46.314081 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbdc6467-c69e-42eb-840b-36c3beb86fb5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:46.314552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbdc6467-c69e-42eb-840b-36c3beb86fb5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:46.315236 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbdc6467-c69e-42eb-840b-36c3beb86fb5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:46.315752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbdc6467-c69e-42eb-840b-36c3beb86fb5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:46.316202 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbdc6467-c69e-42eb-840b-36c3beb86fb5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:46.448020 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bbdc6467-c69e-42eb-840b-36c3beb86fb5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:46.456194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bbdc6467-c69e-42eb-840b-36c3beb86fb5 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1721 microversion: 2.1 time: 0.154136 Oct 20 09:04:46.456755 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 316/641] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:46 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1721 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:46.601317 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0372651d-34ca-4a48-a234-b736213d90f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:46.601850 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0372651d-34ca-4a48-a234-b736213d90f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:46.608930 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0372651d-34ca-4a48-a234-b736213d90f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:46.609412 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0372651d-34ca-4a48-a234-b736213d90f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:46.609846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0372651d-34ca-4a48-a234-b736213d90f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:46.610557 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0372651d-34ca-4a48-a234-b736213d90f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:46.611013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0372651d-34ca-4a48-a234-b736213d90f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:46.611436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0372651d-34ca-4a48-a234-b736213d90f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:46.815583 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0372651d-34ca-4a48-a234-b736213d90f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:46.823776 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0372651d-34ca-4a48-a234-b736213d90f7 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1557 microversion: 2.1 time: 0.226392 Oct 20 09:04:46.824343 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 326/642] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:46 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1557 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:46.847503 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f980b8b3-f0f3-451a-aa90-55a3957c344e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:46.847983 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f980b8b3-f0f3-451a-aa90-55a3957c344e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:46.853876 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f980b8b3-f0f3-451a-aa90-55a3957c344e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:46.854288 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f980b8b3-f0f3-451a-aa90-55a3957c344e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:46.855621 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f980b8b3-f0f3-451a-aa90-55a3957c344e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:46.856334 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f980b8b3-f0f3-451a-aa90-55a3957c344e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:46.857133 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f980b8b3-f0f3-451a-aa90-55a3957c344e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:46.857562 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f980b8b3-f0f3-451a-aa90-55a3957c344e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:46.942338 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f980b8b3-f0f3-451a-aa90-55a3957c344e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.098630 Oct 20 09:04:46.943185 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 317/643] 10.222.0.36 () {58 vars in 1275 bytes} [Thu Oct 20 09:04:46 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-interface => generated 284 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:47.479873 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-16283e1f-4d5d-4a37-ad11-ed1786f940c0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:47.480477 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-16283e1f-4d5d-4a37-ad11-ed1786f940c0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:47.486330 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16283e1f-4d5d-4a37-ad11-ed1786f940c0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:47.486842 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16283e1f-4d5d-4a37-ad11-ed1786f940c0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:47.487275 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16283e1f-4d5d-4a37-ad11-ed1786f940c0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:47.487962 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16283e1f-4d5d-4a37-ad11-ed1786f940c0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:47.488522 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16283e1f-4d5d-4a37-ad11-ed1786f940c0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:47.488947 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16283e1f-4d5d-4a37-ad11-ed1786f940c0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:47.621233 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-16283e1f-4d5d-4a37-ad11-ed1786f940c0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:47.630761 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-16283e1f-4d5d-4a37-ad11-ed1786f940c0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1721 microversion: 2.1 time: 0.154940 Oct 20 09:04:47.631606 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 327/644] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:47 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1721 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:04:48.257481 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-f24730e5-ddb3-441a-9576-e90414d46b66 req-138fdab1-8f14-4e9c-b2a5-e1086bbf4703 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "06d8b443-0b7d-4287-8469-eb4a35863289", "tag": "db1c0050-0582-4d57-ba24-66fe0c293b1e"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:48.267926 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-f24730e5-ddb3-441a-9576-e90414d46b66 req-138fdab1-8f14-4e9c-b2a5-e1086bbf4703 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:48.268489 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-f24730e5-ddb3-441a-9576-e90414d46b66 req-138fdab1-8f14-4e9c-b2a5-e1086bbf4703 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:48.268922 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-f24730e5-ddb3-441a-9576-e90414d46b66 req-138fdab1-8f14-4e9c-b2a5-e1086bbf4703 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:48.281076 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-f24730e5-ddb3-441a-9576-e90414d46b66 req-138fdab1-8f14-4e9c-b2a5-e1086bbf4703 service nova] Creating event network-changed:db1c0050-0582-4d57-ba24-66fe0c293b1e for instance 06d8b443-0b7d-4287-8469-eb4a35863289 on hv-cinder-maste Oct 20 09:04:48.286714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-f24730e5-ddb3-441a-9576-e90414d46b66 req-138fdab1-8f14-4e9c-b2a5-e1086bbf4703 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033476 Oct 20 09:04:48.287569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 318/645] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:48.653505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fe39665f-8b1d-41d0-9cbe-f0bdc404de6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:48.654110 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fe39665f-8b1d-41d0-9cbe-f0bdc404de6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:48.660742 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fe39665f-8b1d-41d0-9cbe-f0bdc404de6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:48.661459 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fe39665f-8b1d-41d0-9cbe-f0bdc404de6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:48.661892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fe39665f-8b1d-41d0-9cbe-f0bdc404de6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:48.662830 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fe39665f-8b1d-41d0-9cbe-f0bdc404de6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:48.663297 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fe39665f-8b1d-41d0-9cbe-f0bdc404de6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:48.663757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fe39665f-8b1d-41d0-9cbe-f0bdc404de6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:48.683505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:04:48.805169 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-fe39665f-8b1d-41d0-9cbe-f0bdc404de6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:48.813912 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-fe39665f-8b1d-41d0-9cbe-f0bdc404de6b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1721 microversion: 2.1 time: 0.164263 Oct 20 09:04:48.814503 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 328/646] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:48 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1721 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:49.837470 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c7727c8b-d451-49d4-b713-2076a48a4f06 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:49.838196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c7727c8b-d451-49d4-b713-2076a48a4f06 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:49.845224 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c7727c8b-d451-49d4-b713-2076a48a4f06 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:49.845646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c7727c8b-d451-49d4-b713-2076a48a4f06 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:49.846080 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c7727c8b-d451-49d4-b713-2076a48a4f06 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:49.846786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c7727c8b-d451-49d4-b713-2076a48a4f06 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:49.847239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c7727c8b-d451-49d4-b713-2076a48a4f06 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:49.847662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c7727c8b-d451-49d4-b713-2076a48a4f06 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:49.981801 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c7727c8b-d451-49d4-b713-2076a48a4f06 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:49.989923 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c7727c8b-d451-49d4-b713-2076a48a4f06 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1721 microversion: 2.1 time: 0.156659 Oct 20 09:04:49.990516 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 319/647] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:49 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1721 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:51.014189 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d1cb6c91-7674-4b20-a557-266fa2d89a2c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:51.014837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d1cb6c91-7674-4b20-a557-266fa2d89a2c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:51.021498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cb6c91-7674-4b20-a557-266fa2d89a2c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:51.021970 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cb6c91-7674-4b20-a557-266fa2d89a2c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:51.022602 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cb6c91-7674-4b20-a557-266fa2d89a2c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:51.023254 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cb6c91-7674-4b20-a557-266fa2d89a2c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:51.023699 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cb6c91-7674-4b20-a557-266fa2d89a2c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:51.024230 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d1cb6c91-7674-4b20-a557-266fa2d89a2c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:51.163219 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d1cb6c91-7674-4b20-a557-266fa2d89a2c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:04:51.171616 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d1cb6c91-7674-4b20-a557-266fa2d89a2c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1727 microversion: 2.1 time: 0.162040 Oct 20 09:04:51.172185 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 329/648] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:04:51 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:04:51.194152 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fc07e43c-2356-4fd6-b54e-05c1f90deb85 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:51.194690 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-fc07e43c-2356-4fd6-b54e-05c1f90deb85 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:51.202221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc07e43c-2356-4fd6-b54e-05c1f90deb85 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:51.202791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc07e43c-2356-4fd6-b54e-05c1f90deb85 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:51.203142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc07e43c-2356-4fd6-b54e-05c1f90deb85 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:51.203836 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc07e43c-2356-4fd6-b54e-05c1f90deb85 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:51.204293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc07e43c-2356-4fd6-b54e-05c1f90deb85 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:51.204726 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fc07e43c-2356-4fd6-b54e-05c1f90deb85 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:51.284825 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fc07e43c-2356-4fd6-b54e-05c1f90deb85 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.094453 Oct 20 09:04:51.285418 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 320/649] 10.222.0.36 () {58 vars in 1275 bytes} [Thu Oct 20 09:04:51 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-interface => generated 285 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:52.170949 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-00908865-36f6-41e8-b9d4-a2232769bcb1 req-0a0e1fce-4dc0-4f34-95a5-5e771da596dc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ea0966d3-1311-4690-a69f-d3df935a76e1", "tag": "b9bd9664-f1f8-415b-84b5-9c0f08f3fc00"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:52.188030 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-00908865-36f6-41e8-b9d4-a2232769bcb1 req-0a0e1fce-4dc0-4f34-95a5-5e771da596dc service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:52.188504 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-00908865-36f6-41e8-b9d4-a2232769bcb1 req-0a0e1fce-4dc0-4f34-95a5-5e771da596dc service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:52.188927 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-00908865-36f6-41e8-b9d4-a2232769bcb1 req-0a0e1fce-4dc0-4f34-95a5-5e771da596dc service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:52.200775 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-00908865-36f6-41e8-b9d4-a2232769bcb1 req-0a0e1fce-4dc0-4f34-95a5-5e771da596dc service nova] Creating event network-changed:b9bd9664-f1f8-415b-84b5-9c0f08f3fc00 for instance ea0966d3-1311-4690-a69f-d3df935a76e1 on hv-cinder-maste Oct 20 09:04:52.206051 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-00908865-36f6-41e8-b9d4-a2232769bcb1 req-0a0e1fce-4dc0-4f34-95a5-5e771da596dc service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041456 Oct 20 09:04:52.206680 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 330/650] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:04:54.223565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-fdd69483-7fe4-4397-b33c-1df6dd7c2fa1 req-ad2af79d-48ce-403f-abfb-08fb3781f396 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ea0966d3-1311-4690-a69f-d3df935a76e1", "tag": "b9bd9664-f1f8-415b-84b5-9c0f08f3fc00"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:04:54.233989 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-fdd69483-7fe4-4397-b33c-1df6dd7c2fa1 req-ad2af79d-48ce-403f-abfb-08fb3781f396 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:54.234562 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-fdd69483-7fe4-4397-b33c-1df6dd7c2fa1 req-ad2af79d-48ce-403f-abfb-08fb3781f396 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:54.235065 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-fdd69483-7fe4-4397-b33c-1df6dd7c2fa1 req-ad2af79d-48ce-403f-abfb-08fb3781f396 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:54.246829 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-fdd69483-7fe4-4397-b33c-1df6dd7c2fa1 req-ad2af79d-48ce-403f-abfb-08fb3781f396 service nova] Creating event network-changed:b9bd9664-f1f8-415b-84b5-9c0f08f3fc00 for instance ea0966d3-1311-4690-a69f-d3df935a76e1 on hv-cinder-maste Oct 20 09:04:54.252182 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-fdd69483-7fe4-4397-b33c-1df6dd7c2fa1 req-ad2af79d-48ce-403f-abfb-08fb3781f396 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034652 Oct 20 09:04:54.252964 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 321/651] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:04:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (3 switches on core 0) Oct 20 09:04:58.535595 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6de6a5b1-f9bc-480d-84bc-f1ae60c27661 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:58.536249 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6de6a5b1-f9bc-480d-84bc-f1ae60c27661 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:58.542274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6de6a5b1-f9bc-480d-84bc-f1ae60c27661 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:58.542788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6de6a5b1-f9bc-480d-84bc-f1ae60c27661 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:58.543249 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6de6a5b1-f9bc-480d-84bc-f1ae60c27661 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:58.543930 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6de6a5b1-f9bc-480d-84bc-f1ae60c27661 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:58.544373 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6de6a5b1-f9bc-480d-84bc-f1ae60c27661 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:58.544798 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6de6a5b1-f9bc-480d-84bc-f1ae60c27661 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:58.573774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6de6a5b1-f9bc-480d-84bc-f1ae60c27661 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments/796139d0-582f-43cf-a128-27d67f6063fb" status: 200 len: 194 microversion: 2.20 time: 0.043666 Oct 20 09:04:58.574962 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 331/652] 10.222.0.36 () {60 vars in 1454 bytes} [Thu Oct 20 09:04:58 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments/796139d0-582f-43cf-a128-27d67f6063fb => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 09:04:58.645826 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fca8759e-e60c-4802-98df-7b8edb343033 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:58.646456 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-fca8759e-e60c-4802-98df-7b8edb343033 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:58.653059 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fca8759e-e60c-4802-98df-7b8edb343033 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:58.653539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fca8759e-e60c-4802-98df-7b8edb343033 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:58.653968 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fca8759e-e60c-4802-98df-7b8edb343033 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:58.654680 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fca8759e-e60c-4802-98df-7b8edb343033 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:58.655129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fca8759e-e60c-4802-98df-7b8edb343033 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:58.655545 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fca8759e-e60c-4802-98df-7b8edb343033 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:58.738653 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-fca8759e-e60c-4802-98df-7b8edb343033 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lazy-loading 'services' on Instance uuid f8d61907-24ca-4e74-917e-d218d0f89557 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:04:58.849034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fca8759e-e60c-4802-98df-7b8edb343033 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "DELETE /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments/796139d0-582f-43cf-a128-27d67f6063fb" status: 202 len: 0 microversion: 2.20 time: 0.208158 Oct 20 09:04:58.849673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 322/653] 10.222.0.36 () {60 vars in 1457 bytes} [Thu Oct 20 09:04:58 2022] DELETE /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments/796139d0-582f-43cf-a128-27d67f6063fb => generated 0 bytes in 210 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:04:58.860352 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-362cf87c-ccc7-4226-91ca-a85ff46ddb02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:58.862956 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-362cf87c-ccc7-4226-91ca-a85ff46ddb02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:58.870868 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-362cf87c-ccc7-4226-91ca-a85ff46ddb02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:58.872084 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-362cf87c-ccc7-4226-91ca-a85ff46ddb02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:58.872666 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-362cf87c-ccc7-4226-91ca-a85ff46ddb02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:58.873672 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-362cf87c-ccc7-4226-91ca-a85ff46ddb02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:58.874178 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-362cf87c-ccc7-4226-91ca-a85ff46ddb02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:58.875265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-362cf87c-ccc7-4226-91ca-a85ff46ddb02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:58.908636 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-362cf87c-ccc7-4226-91ca-a85ff46ddb02 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054437 Oct 20 09:04:58.909320 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 332/654] 10.222.0.36 () {60 vars in 1343 bytes} [Thu Oct 20 09:04:58 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 09:04:59.923067 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-38d1a655-1aee-49da-ba8e-cc96958573ab tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:04:59.925318 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-38d1a655-1aee-49da-ba8e-cc96958573ab tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:04:59.931356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38d1a655-1aee-49da-ba8e-cc96958573ab tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:59.931830 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38d1a655-1aee-49da-ba8e-cc96958573ab tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:59.932256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38d1a655-1aee-49da-ba8e-cc96958573ab tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:59.932924 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38d1a655-1aee-49da-ba8e-cc96958573ab tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:04:59.933364 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38d1a655-1aee-49da-ba8e-cc96958573ab tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:04:59.934427 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-38d1a655-1aee-49da-ba8e-cc96958573ab tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:04:59.967256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-38d1a655-1aee-49da-ba8e-cc96958573ab tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049669 Oct 20 09:04:59.967799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 323/655] 10.222.0.36 () {60 vars in 1343 bytes} [Thu Oct 20 09:04:59 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 09:05:00.980678 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e15c42f6-7ad1-4179-95e6-c4a67ff7fc2f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:00.982934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-e15c42f6-7ad1-4179-95e6-c4a67ff7fc2f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:00.989133 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e15c42f6-7ad1-4179-95e6-c4a67ff7fc2f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:00.989613 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e15c42f6-7ad1-4179-95e6-c4a67ff7fc2f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:00.990048 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e15c42f6-7ad1-4179-95e6-c4a67ff7fc2f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:00.990752 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e15c42f6-7ad1-4179-95e6-c4a67ff7fc2f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:00.991198 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e15c42f6-7ad1-4179-95e6-c4a67ff7fc2f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:00.991613 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e15c42f6-7ad1-4179-95e6-c4a67ff7fc2f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:01.021396 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e15c42f6-7ad1-4179-95e6-c4a67ff7fc2f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.045374 Oct 20 09:05:01.022187 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 333/656] 10.222.0.36 () {60 vars in 1343 bytes} [Thu Oct 20 09:05:00 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:01.087489 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c2326d18-bdcf-4c69-879f-023fdd671f29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:01.087980 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c2326d18-bdcf-4c69-879f-023fdd671f29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:01.090648 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=59,nova_api:INSERT=3 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:01.095710 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2326d18-bdcf-4c69-879f-023fdd671f29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:01.096187 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2326d18-bdcf-4c69-879f-023fdd671f29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:01.096619 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2326d18-bdcf-4c69-879f-023fdd671f29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:01.097293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2326d18-bdcf-4c69-879f-023fdd671f29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:01.097739 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2326d18-bdcf-4c69-879f-023fdd671f29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:01.098156 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c2326d18-bdcf-4c69-879f-023fdd671f29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:01.125006 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c2326d18-bdcf-4c69-879f-023fdd671f29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:05:01.169226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=98,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:01.203718 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c2326d18-bdcf-4c69-879f-023fdd671f29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "DELETE /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 204 len: 0 microversion: 2.20 time: 0.119823 Oct 20 09:05:01.204835 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 324/657] 10.222.0.36 () {60 vars in 1280 bytes} [Thu Oct 20 09:05:01 2022] DELETE /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 20 09:05:01.212737 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:01.213348 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:01.219780 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:01.220267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:01.220707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:01.221397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:01.221853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:01.222285 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:01.393659 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:01.401954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:01.403483 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:01.404669 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5bc55548-7333-4a8a-acd8-ab9927c41e4e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.195837 Oct 20 09:05:01.405221 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 334/658] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:01 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:02.431115 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:02.431840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:02.439362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:02.439814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:02.440318 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:02.441086 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:02.441516 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:02.441934 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:02.656005 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:02.665992 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:02.667454 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:02.668645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-43e76bac-180e-4da5-b53d-85c2ae139c53 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.241854 Oct 20 09:05:02.673693 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 325/659] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:02 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:03.693381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:03.694640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:03.701542 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:03.702029 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:03.702458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:03.703246 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:03.703632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:03.704048 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:04.071974 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:04.080943 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:04.082358 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:04.084016 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bb0a364a-f277-47a7-af43-bef03c4c77f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1720 microversion: 2.20 time: 0.394714 Oct 20 09:05:04.084565 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 335/660] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:03 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1720 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:04.959104 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-6c4f260b-064e-4ba2-a1b1-1f091d821104 req-bade450b-6a38-4822-80af-8eec1e720773 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f8d61907-24ca-4e74-917e-d218d0f89557", "name": "network-vif-deleted", "tag": "6ba2009e-08be-4c65-a1d5-5e9548c8d407"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:04.979088 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6c4f260b-064e-4ba2-a1b1-1f091d821104 req-bade450b-6a38-4822-80af-8eec1e720773 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:04.979580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6c4f260b-064e-4ba2-a1b1-1f091d821104 req-bade450b-6a38-4822-80af-8eec1e720773 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:04.980049 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6c4f260b-064e-4ba2-a1b1-1f091d821104 req-bade450b-6a38-4822-80af-8eec1e720773 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:04.993913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-6c4f260b-064e-4ba2-a1b1-1f091d821104 req-bade450b-6a38-4822-80af-8eec1e720773 service nova] Creating event network-vif-deleted:6ba2009e-08be-4c65-a1d5-5e9548c8d407 for instance f8d61907-24ca-4e74-917e-d218d0f89557 on hv-cinder-maste Oct 20 09:05:04.999827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-6c4f260b-064e-4ba2-a1b1-1f091d821104 req-bade450b-6a38-4822-80af-8eec1e720773 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045646 Oct 20 09:05:05.000782 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 326/661] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:05:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:05.110097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:05.110936 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:05.118823 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:05.119321 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:05.119759 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:05.120474 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:05.121541 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:05.122223 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:05.159897 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:05.160761 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:05.161797 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:05.162432 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lazy-loading 'fault' on Instance uuid f8d61907-24ca-4e74-917e-d218d0f89557 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:05:05.241195 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:05.250249 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:05.252087 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:05.253526 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e0342d64-0ebd-43a9-a09e-1e877c47a063 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 200 len: 1370 microversion: 2.20 time: 0.148180 Oct 20 09:05:05.266920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 336/662] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:05 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 1370 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:06.281103 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:06.281933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:06.288462 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:06.288948 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:06.289387 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:06.290086 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:06.290565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:06.290991 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:06.305600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] HTTP exception thrown: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Request method failure captured: Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 HTTP/1.1 Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ45rraxKtmAczlSJFs0dTqdOsnHgw3sd7cCCv7-jWXet363TlKZqjHREJ1szDoP0_9r30G6izG2SMxnb9AdlZuYz2zkipPGMfoApkn6nORm837Fs5_N4Zn-_5QRzx55zhX2116SyS1aHgjJH8Dc90kajhoUZL99KLxKxA6o6SKuG8g Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Openstack-Nova-Api-Version: 2.20 Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:05:06.307294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_869b5cf3a9264fd3b70ae656d7814cfb", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}] Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: fd9733d1bb2b4520b70090542615ddf9 Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:05:06.309485 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': 'f8d61907-24ca-4e74-917e-d218d0f89557'} Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:05:06.310226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:05:06.311498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:05:06.314390 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Returning 404 to user: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:05:06.315293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-9f77a9c1-01e2-460b-a0d8-50a024714b07 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 404 len: 111 microversion: 2.20 time: 0.039140 Oct 20 09:05:06.316186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 327/663] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:06 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 20 09:05:06.325566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f32742f1-b1ab-4860-a7e4-ab0d8dd1c9c4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '.version_select at 0x7f5afbac7670>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:06.346706 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f32742f1-b1ab-4860-a7e4-ab0d8dd1c9c4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-121686616" status: 204 len: 0 microversion: 2.20 time: 0.025872 Oct 20 09:05:06.347619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 337/664] 10.222.0.36 () {60 vars in 1259 bytes} [Thu Oct 20 09:05:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-121686616 => generated 0 bytes in 27 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 20 09:05:07.109631 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ba0a1b3d-b301-4986-9f90-20b07b4c8924 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Action: 'create', calling method: .version_select at 0x7f5afd24c1f0>, body: {"keypair": {"name": "tempest-keypair-1573757010", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBENezA1GDTDe29M8CT3qvKAN309JMXAMbu5kfrtYojjSDyrLFMCYmDMklcIoLCy+cOfXLRyYFsuS+daD/sI6txxI/pqih+7mvVixb/IgBurCbtNqS3ddiU8oloLxlQpicQ=="}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:07.126222 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-ba0a1b3d-b301-4986-9f90-20b07b4c8924 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Getting quotas for project 869b5cf3a9264fd3b70ae656d7814cfb. Resources: {'key_pairs'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:05:07.130460 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-ba0a1b3d-b301-4986-9f90-20b07b4c8924 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Getting quotas for user fd9733d1bb2b4520b70090542615ddf9 and project 869b5cf3a9264fd3b70ae656d7814cfb. Resources: {'key_pairs'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:05:07.154382 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ba0a1b3d-b301-4986-9f90-20b07b4c8924 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.049613 Oct 20 09:05:07.155150 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 328/665] 10.222.0.36 () {62 vars in 1200 bytes} [Thu Oct 20 09:05:07 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 51 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 20 09:05:08.815978 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-2110425972", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8e03c888-668e-46f5-b105-1902608fb69d"}], "security_groups": [{"name": "tempest-securitygroup--1314945510"}], "key_name": "tempest-keypair-1573757010", "user_data": "}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:08.948022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] validate_networks() for [('8e03c888-668e-46f5-b105-1902608fb69d', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:05:09.377599 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:05:09.378307 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:05:09.378951 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:05:09.379518 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:05:09.380068 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:05:09.391344 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:09.392428 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:09.393003 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:09.394531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:09.395326 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:09.395894 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:09.417360 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Getting quotas for project 869b5cf3a9264fd3b70ae656d7814cfb. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:05:09.421935 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Getting quotas for user fd9733d1bb2b4520b70090542615ddf9 and project 869b5cf3a9264fd3b70ae656d7814cfb. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:05:09.433784 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:05:09.436255 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:05:09.497134 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-98c22002-5ac5-49f2-af09-8893c5eb9d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.687096 Oct 20 09:05:09.498560 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 338/666] 10.222.0.36 () {62 vars in 1188 bytes} [Thu Oct 20 09:05:08 2022] POST /compute/v2.1/servers => generated 402 bytes in 690 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 20 09:05:09.512716 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f618745c-99d5-4761-b4ad-b9a1c115d884 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:09.513913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f618745c-99d5-4761-b4ad-b9a1c115d884 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:09.627762 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f618745c-99d5-4761-b4ad-b9a1c115d884 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:09.659356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f618745c-99d5-4761-b4ad-b9a1c115d884 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:09.661630 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0b648e26-dd6e-465e-8607-756cfa9f6784 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:09.662364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0b648e26-dd6e-465e-8607-756cfa9f6784 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:09.669651 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b648e26-dd6e-465e-8607-756cfa9f6784 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:09.670632 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b648e26-dd6e-465e-8607-756cfa9f6784 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:09.671460 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b648e26-dd6e-465e-8607-756cfa9f6784 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:09.672380 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b648e26-dd6e-465e-8607-756cfa9f6784 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:09.673045 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b648e26-dd6e-465e-8607-756cfa9f6784 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:09.674382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0b648e26-dd6e-465e-8607-756cfa9f6784 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:09.678164 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f618745c-99d5-4761-b4ad-b9a1c115d884 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:09.681411 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f618745c-99d5-4761-b4ad-b9a1c115d884 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1282 microversion: 2.20 time: 0.176124 Oct 20 09:05:09.687170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 329/667] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:09 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1282 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:09.799098 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-0b648e26-dd6e-465e-8607-756cfa9f6784 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lazy-loading 'services' on Instance uuid 26ae0069-be67-4921-aca3-4cfae5214cf5 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:05:09.889649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0b648e26-dd6e-465e-8607-756cfa9f6784 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "DELETE /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/os-volume_attachments/bcaf282a-ef82-4a51-85a4-da738df00b21" status: 202 len: 0 microversion: 2.1 time: 0.229494 Oct 20 09:05:09.903210 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 339/668] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:05:09 2022] DELETE /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/os-volume_attachments/bcaf282a-ef82-4a51-85a4-da738df00b21 => generated 0 bytes in 248 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:05:10.725738 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:10.726448 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:10.742441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:10.746321 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:10.747916 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:10.750003 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:10.750744 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:10.752177 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:10.837759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:10.847101 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:10.848815 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:10.850147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f1bf077e-3b29-4e30-9e96-e8a56574f309 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1278 microversion: 2.20 time: 0.135203 Oct 20 09:05:10.851246 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 330/669] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:10 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1278 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:11.882185 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:11.884309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:11.901301 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:11.902200 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:11.902927 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:11.903865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:11.905101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:11.905990 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:12.068865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:12.081466 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:12.084003 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:12.085598 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a49a853d-27e9-4aa4-9f5d-42e404026464 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1342 microversion: 2.20 time: 0.209499 Oct 20 09:05:12.086906 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 340/670] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:11 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1342 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:12.268737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-812ce5d3-278e-4194-b828-73a71f53c7d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:12.282630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-812ce5d3-278e-4194-b828-73a71f53c7d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:12.283327 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-812ce5d3-278e-4194-b828-73a71f53c7d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:12.283928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-812ce5d3-278e-4194-b828-73a71f53c7d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:12.336929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-812ce5d3-278e-4194-b828-73a71f53c7d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Going to try to stop instance {{(pid=119068) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 20 09:05:12.449511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-812ce5d3-278e-4194-b828-73a71f53c7d6 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/action" status: 202 len: 0 microversion: 2.1 time: 0.183958 Oct 20 09:05:12.450747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 331/671] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:05:12 2022] POST /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/action => generated 0 bytes in 186 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:05:12.458853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9d53c7db-c47c-4253-b9d2-7975ef9df022 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:12.459749 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9d53c7db-c47c-4253-b9d2-7975ef9df022 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:12.495762 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d53c7db-c47c-4253-b9d2-7975ef9df022 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:12.497296 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d53c7db-c47c-4253-b9d2-7975ef9df022 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:12.498031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d53c7db-c47c-4253-b9d2-7975ef9df022 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:12.499165 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d53c7db-c47c-4253-b9d2-7975ef9df022 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:12.499878 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d53c7db-c47c-4253-b9d2-7975ef9df022 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:12.500461 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9d53c7db-c47c-4253-b9d2-7975ef9df022 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:12.942164 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9d53c7db-c47c-4253-b9d2-7975ef9df022 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:12.950808 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9d53c7db-c47c-4253-b9d2-7975ef9df022 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1677 microversion: 2.1 time: 0.495967 Oct 20 09:05:12.951399 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 341/672] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:12 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1677 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:13.117209 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:13.119425 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:13.127737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:13.128351 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:13.128793 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:13.129484 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:13.129932 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:13.130373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:13.206518 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:13.216381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:13.218012 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:13.219339 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e5b7749c-0982-43dc-94e7-a932daf36b39 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1340 microversion: 2.20 time: 0.107671 Oct 20 09:05:13.220120 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 332/673] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:13 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1340 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:13.979112 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6cf76e54-71b7-44db-9c32-3c4a2db4844c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:13.979672 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6cf76e54-71b7-44db-9c32-3c4a2db4844c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:13.993265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6cf76e54-71b7-44db-9c32-3c4a2db4844c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:13.993711 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6cf76e54-71b7-44db-9c32-3c4a2db4844c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:13.994146 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6cf76e54-71b7-44db-9c32-3c4a2db4844c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:13.994873 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6cf76e54-71b7-44db-9c32-3c4a2db4844c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:13.995322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6cf76e54-71b7-44db-9c32-3c4a2db4844c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:13.995779 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6cf76e54-71b7-44db-9c32-3c4a2db4844c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:14.249381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:14.251859 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:14.259627 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:14.260436 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:14.260991 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:14.262494 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:14.263182 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:14.263718 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:14.301720 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6cf76e54-71b7-44db-9c32-3c4a2db4844c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:14.310417 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6cf76e54-71b7-44db-9c32-3c4a2db4844c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1677 microversion: 2.1 time: 0.335160 Oct 20 09:05:14.321703 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 342/674] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:13 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1677 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:14.444207 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:14.454247 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:14.455800 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:14.457012 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3e5952fc-dcb9-4d4f-9b31-2dd7e1a3f76c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1408 microversion: 2.20 time: 0.214570 Oct 20 09:05:14.458085 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 333/675] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:14 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1408 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:15.049631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-19a33985-fe77-4078-92d3-a82db80427ab req-5962b855-fcad-465a-892d-d69ee49e34d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7facb789-628a-44aa-b0ec-7f6aee5617dd", "tag": "f17302af-3ee4-46d2-a6fe-a973b183f4e4"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:15.059876 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-19a33985-fe77-4078-92d3-a82db80427ab req-5962b855-fcad-465a-892d-d69ee49e34d2 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:15.060621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-19a33985-fe77-4078-92d3-a82db80427ab req-5962b855-fcad-465a-892d-d69ee49e34d2 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:15.061214 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-19a33985-fe77-4078-92d3-a82db80427ab req-5962b855-fcad-465a-892d-d69ee49e34d2 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:15.078963 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-19a33985-fe77-4078-92d3-a82db80427ab req-5962b855-fcad-465a-892d-d69ee49e34d2 service nova] Creating event network-changed:f17302af-3ee4-46d2-a6fe-a973b183f4e4 for instance 7facb789-628a-44aa-b0ec-7f6aee5617dd on hv-cinder-maste Oct 20 09:05:15.084313 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-19a33985-fe77-4078-92d3-a82db80427ab req-5962b855-fcad-465a-892d-d69ee49e34d2 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038297 Oct 20 09:05:15.085540 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 343/676] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:05:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:15.347434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1e8e0fe9-fbf6-4622-abfd-69e4e9efdd06 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:15.349513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1e8e0fe9-fbf6-4622-abfd-69e4e9efdd06 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:15.358925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e8e0fe9-fbf6-4622-abfd-69e4e9efdd06 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:15.359624 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e8e0fe9-fbf6-4622-abfd-69e4e9efdd06 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:15.360232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e8e0fe9-fbf6-4622-abfd-69e4e9efdd06 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:15.361119 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e8e0fe9-fbf6-4622-abfd-69e4e9efdd06 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:15.361731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e8e0fe9-fbf6-4622-abfd-69e4e9efdd06 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:15.362909 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e8e0fe9-fbf6-4622-abfd-69e4e9efdd06 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:15.483440 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:15.484174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:15.486076 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=54,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:15.493263 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:15.494054 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:15.494681 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:15.496222 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:15.496842 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:15.497427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:15.591984 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1e8e0fe9-fbf6-4622-abfd-69e4e9efdd06 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:15.601055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1e8e0fe9-fbf6-4622-abfd-69e4e9efdd06 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1677 microversion: 2.1 time: 0.259772 Oct 20 09:05:15.602360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 334/677] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:15 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1677 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:15.789124 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:15.802774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:15.805538 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:15.807444 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-af58a551-6b83-423d-92e5-b69b69ee41e9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1408 microversion: 2.20 time: 0.328870 Oct 20 09:05:15.831787 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 344/678] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:15 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1408 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:05:16.625676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5a443c4b-8088-4e08-b213-100b33e7e8d8 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:16.627062 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5a443c4b-8088-4e08-b213-100b33e7e8d8 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:16.636189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5a443c4b-8088-4e08-b213-100b33e7e8d8 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:16.637796 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5a443c4b-8088-4e08-b213-100b33e7e8d8 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:16.638456 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5a443c4b-8088-4e08-b213-100b33e7e8d8 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:16.639439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5a443c4b-8088-4e08-b213-100b33e7e8d8 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:16.640260 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5a443c4b-8088-4e08-b213-100b33e7e8d8 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:16.640907 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5a443c4b-8088-4e08-b213-100b33e7e8d8 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:16.800149 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5a443c4b-8088-4e08-b213-100b33e7e8d8 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:16.811255 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5a443c4b-8088-4e08-b213-100b33e7e8d8 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1677 microversion: 2.1 time: 0.189420 Oct 20 09:05:16.814143 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 335/679] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:16 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1677 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:16.835972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:16.837169 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:16.845111 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:16.845600 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:16.846037 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:16.846772 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:16.847225 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:16.847649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:17.137013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:17.146204 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:17.148105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:17.149419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c7104d1b-f01c-4e4b-93e7-d5c22c6b3797 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1408 microversion: 2.20 time: 0.318455 Oct 20 09:05:17.163590 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 345/680] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:16 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1408 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:17.837105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f890219d-64c5-415b-be12-6e597d43a29a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:17.837722 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f890219d-64c5-415b-be12-6e597d43a29a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:17.847332 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f890219d-64c5-415b-be12-6e597d43a29a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:17.847834 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f890219d-64c5-415b-be12-6e597d43a29a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:17.848274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f890219d-64c5-415b-be12-6e597d43a29a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:17.848989 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f890219d-64c5-415b-be12-6e597d43a29a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:17.849440 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f890219d-64c5-415b-be12-6e597d43a29a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:17.850069 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f890219d-64c5-415b-be12-6e597d43a29a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:18.005272 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f890219d-64c5-415b-be12-6e597d43a29a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:18.015686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f890219d-64c5-415b-be12-6e597d43a29a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1677 microversion: 2.1 time: 0.182337 Oct 20 09:05:18.023017 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 336/681] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:17 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1677 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:18.176131 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:18.176744 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:18.184126 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:18.184605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:18.185031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:18.185723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:18.186165 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:18.186605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:18.381593 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:18.389965 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:18.392309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:18.393689 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a94bd10b-d85e-4036-863c-cd47adab9c8d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1408 microversion: 2.20 time: 0.221574 Oct 20 09:05:18.394487 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 346/682] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:18 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1408 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:19.038870 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b6ea3990-0683-41eb-a7bc-eae9bd5e2483 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:19.039482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b6ea3990-0683-41eb-a7bc-eae9bd5e2483 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:19.051641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b6ea3990-0683-41eb-a7bc-eae9bd5e2483 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:19.052405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b6ea3990-0683-41eb-a7bc-eae9bd5e2483 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:19.053020 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b6ea3990-0683-41eb-a7bc-eae9bd5e2483 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:19.053901 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b6ea3990-0683-41eb-a7bc-eae9bd5e2483 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:19.054552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b6ea3990-0683-41eb-a7bc-eae9bd5e2483 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:19.055135 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b6ea3990-0683-41eb-a7bc-eae9bd5e2483 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:19.196443 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b6ea3990-0683-41eb-a7bc-eae9bd5e2483 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:19.204448 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b6ea3990-0683-41eb-a7bc-eae9bd5e2483 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1677 microversion: 2.1 time: 0.169706 Oct 20 09:05:19.206328 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 337/683] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:19 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1677 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:19.421441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:19.422034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:19.428333 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:19.431125 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:19.434272 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:19.435101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:19.436199 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:19.437011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:19.437887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:19.439802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=6 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:19.445865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=98,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:19.592885 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:19.602812 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:19.604251 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:19.605990 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c84f8109-7624-4b65-b882-be4981b577b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1408 microversion: 2.20 time: 0.191356 Oct 20 09:05:19.606567 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 347/684] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:19 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1408 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:20.231859 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d40576bd-6f3e-4c9f-9f57-bc3967db04b2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:20.232593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d40576bd-6f3e-4c9f-9f57-bc3967db04b2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:20.235678 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=6 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:20.249744 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d40576bd-6f3e-4c9f-9f57-bc3967db04b2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:20.250239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d40576bd-6f3e-4c9f-9f57-bc3967db04b2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:20.250838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d40576bd-6f3e-4c9f-9f57-bc3967db04b2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:20.251547 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d40576bd-6f3e-4c9f-9f57-bc3967db04b2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:20.252058 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d40576bd-6f3e-4c9f-9f57-bc3967db04b2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:20.252489 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d40576bd-6f3e-4c9f-9f57-bc3967db04b2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:20.408725 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d40576bd-6f3e-4c9f-9f57-bc3967db04b2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:20.416658 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d40576bd-6f3e-4c9f-9f57-bc3967db04b2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1677 microversion: 2.1 time: 0.188683 Oct 20 09:05:20.417533 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 338/685] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:20 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1677 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:20.631163 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:20.632446 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:20.639609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:20.640331 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:20.640983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:20.641869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:20.642531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:20.642955 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:20.789791 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:20.799370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:20.800843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:20.802017 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-511f9488-2793-4e44-8233-62117ec3bef5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1408 microversion: 2.20 time: 0.174632 Oct 20 09:05:20.802625 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 348/686] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:20 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1408 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:21.394793 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-7b18e0d4-7204-44d4-89d4-f01a02716bc6 req-6f5658f0-91d6-48a0-b77c-c6a773da38b3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7facb789-628a-44aa-b0ec-7f6aee5617dd", "name": "network-vif-plugged", "status": "completed", "tag": "f17302af-3ee4-46d2-a6fe-a973b183f4e4"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:21.407849 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-7b18e0d4-7204-44d4-89d4-f01a02716bc6 req-6f5658f0-91d6-48a0-b77c-c6a773da38b3 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:21.408574 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-7b18e0d4-7204-44d4-89d4-f01a02716bc6 req-6f5658f0-91d6-48a0-b77c-c6a773da38b3 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:21.409232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-7b18e0d4-7204-44d4-89d4-f01a02716bc6 req-6f5658f0-91d6-48a0-b77c-c6a773da38b3 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:21.437791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-7b18e0d4-7204-44d4-89d4-f01a02716bc6 req-6f5658f0-91d6-48a0-b77c-c6a773da38b3 service nova] Creating event network-vif-plugged:f17302af-3ee4-46d2-a6fe-a973b183f4e4 for instance 7facb789-628a-44aa-b0ec-7f6aee5617dd on hv-cinder-maste Oct 20 09:05:21.441613 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1cfe4f0b-29fd-4f1d-9fdc-2ce13679d7aa tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:21.442426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1cfe4f0b-29fd-4f1d-9fdc-2ce13679d7aa tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:21.445240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-7b18e0d4-7204-44d4-89d4-f01a02716bc6 req-6f5658f0-91d6-48a0-b77c-c6a773da38b3 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053944 Oct 20 09:05:21.446232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 339/687] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:05:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:21.449660 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1cfe4f0b-29fd-4f1d-9fdc-2ce13679d7aa tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:21.450138 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1cfe4f0b-29fd-4f1d-9fdc-2ce13679d7aa tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:21.450594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1cfe4f0b-29fd-4f1d-9fdc-2ce13679d7aa tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:21.451648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1cfe4f0b-29fd-4f1d-9fdc-2ce13679d7aa tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:21.452500 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1cfe4f0b-29fd-4f1d-9fdc-2ce13679d7aa tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:21.454248 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1cfe4f0b-29fd-4f1d-9fdc-2ce13679d7aa tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:21.634993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1cfe4f0b-29fd-4f1d-9fdc-2ce13679d7aa tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:21.659689 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1cfe4f0b-29fd-4f1d-9fdc-2ce13679d7aa tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1677 microversion: 2.1 time: 0.222001 Oct 20 09:05:21.660799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 349/688] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:21 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1677 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:21.829312 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:21.830189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:21.836357 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:21.837037 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:21.837666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:21.838553 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:21.839267 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:21.839869 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:21.984994 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:21.993983 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:21.996160 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:21.997470 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a1736fa8-193f-4778-91cf-8a50dd6a4e3f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1408 microversion: 2.20 time: 0.172773 Oct 20 09:05:21.998029 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 340/689] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:21 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1408 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:22.684334 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-30c24cf3-a550-462a-b6cb-bd12d76b4fe2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:22.685925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-30c24cf3-a550-462a-b6cb-bd12d76b4fe2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:22.692868 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-30c24cf3-a550-462a-b6cb-bd12d76b4fe2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:22.693581 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-30c24cf3-a550-462a-b6cb-bd12d76b4fe2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:22.694201 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-30c24cf3-a550-462a-b6cb-bd12d76b4fe2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:22.695082 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-30c24cf3-a550-462a-b6cb-bd12d76b4fe2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:22.695529 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-30c24cf3-a550-462a-b6cb-bd12d76b4fe2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:22.696278 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-30c24cf3-a550-462a-b6cb-bd12d76b4fe2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:22.836119 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-30c24cf3-a550-462a-b6cb-bd12d76b4fe2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:22.844778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-30c24cf3-a550-462a-b6cb-bd12d76b4fe2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1677 microversion: 2.1 time: 0.164917 Oct 20 09:05:22.845589 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 350/690] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:22 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1677 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:23.032879 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:23.034223 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:23.047747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:23.048236 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:23.048671 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:23.049373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:23.049818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:23.050250 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:23.226812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:23.237177 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:23.238804 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:23.240587 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-161f541b-e71a-474f-a51f-78f5a028f78c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1408 microversion: 2.20 time: 0.221363 Oct 20 09:05:23.241154 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 341/691] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:23 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1408 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:05:23.868839 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0879d041-ec4c-418f-b2ee-61d02eb6ac46 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:23.870028 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0879d041-ec4c-418f-b2ee-61d02eb6ac46 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:23.876267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0879d041-ec4c-418f-b2ee-61d02eb6ac46 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:23.876739 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0879d041-ec4c-418f-b2ee-61d02eb6ac46 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:23.877360 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0879d041-ec4c-418f-b2ee-61d02eb6ac46 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:23.878038 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0879d041-ec4c-418f-b2ee-61d02eb6ac46 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:23.878527 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0879d041-ec4c-418f-b2ee-61d02eb6ac46 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:23.879153 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0879d041-ec4c-418f-b2ee-61d02eb6ac46 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:24.021276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0879d041-ec4c-418f-b2ee-61d02eb6ac46 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:24.029093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0879d041-ec4c-418f-b2ee-61d02eb6ac46 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1654 microversion: 2.1 time: 0.164222 Oct 20 09:05:24.030271 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 351/692] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:23 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1654 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:24.056096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fe187d28-a1dd-4826-8e16-05c8f4b8d36c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:24.063702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fe187d28-a1dd-4826-8e16-05c8f4b8d36c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:24.063702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fe187d28-a1dd-4826-8e16-05c8f4b8d36c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:24.063702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fe187d28-a1dd-4826-8e16-05c8f4b8d36c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:24.085657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-fe187d28-a1dd-4826-8e16-05c8f4b8d36c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Going to try to start instance {{(pid=119068) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 20 09:05:24.129586 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fe187d28-a1dd-4826-8e16-05c8f4b8d36c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/action" status: 202 len: 0 microversion: 2.1 time: 0.080880 Oct 20 09:05:24.130182 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 342/693] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:05:24 2022] POST /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/action => generated 0 bytes in 82 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:05:24.139210 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c6e6411b-7796-46af-b157-56e64afe74ef tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:24.140830 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c6e6411b-7796-46af-b157-56e64afe74ef tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:24.146924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c6e6411b-7796-46af-b157-56e64afe74ef tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:24.147408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c6e6411b-7796-46af-b157-56e64afe74ef tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:24.147849 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c6e6411b-7796-46af-b157-56e64afe74ef tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:24.148546 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c6e6411b-7796-46af-b157-56e64afe74ef tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:24.148997 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c6e6411b-7796-46af-b157-56e64afe74ef tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:24.149422 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c6e6411b-7796-46af-b157-56e64afe74ef tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:24.193424 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-06a53b74-1fc2-4b57-9023-12a438608d70 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Action: 'create', calling method: .version_select at 0x7f5afb936ee0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1771110173", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAbjVQyA9uaezkLxl8vFSPhN7LjrtG+q/7Kb/EIfiiGuHlTPC9kjCIvrStd8DvnXt/HEuJ0Hmmt6P6RItevk1Nh9xp7Muqifj1vtOi3syKogNJKJlPsngD8bqskArTWw7Q=="}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:24.209118 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-06a53b74-1fc2-4b57-9023-12a438608d70 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Getting quotas for project 020d4019643f4499b5162717fbf7b5b5. Resources: {'key_pairs'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:05:24.212902 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-06a53b74-1fc2-4b57-9023-12a438608d70 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Getting quotas for user 14ddd15ab87a4d0fbea1d6dfa910f8ce and project 020d4019643f4499b5162717fbf7b5b5. Resources: {'key_pairs'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:05:24.237017 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-06a53b74-1fc2-4b57-9023-12a438608d70 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.047737 Oct 20 09:05:24.237522 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 343/694] 10.222.0.36 () {60 vars in 1159 bytes} [Thu Oct 20 09:05:24 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:05:24.267010 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:24.267758 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:24.273910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:24.274762 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:24.275398 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:24.276263 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:24.277065 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:24.277656 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:24.282286 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:24.317487 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c6e6411b-7796-46af-b157-56e64afe74ef tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:24.325953 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c6e6411b-7796-46af-b157-56e64afe74ef tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1663 microversion: 2.1 time: 0.191454 Oct 20 09:05:24.330555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 352/695] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:24 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1663 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:24.331782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1346493045", "imageRef": "a3c54c89-6f3a-4e6c-95d5-184dbc52ed18", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1771110173", "networks": [{"uuid": "c728b9db-6a87-4eb2-b7e6-b2ca9d5060dc"}]}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:24.392492 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] validate_networks() for [('c728b9db-6a87-4eb2-b7e6-b2ca9d5060dc', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:05:24.436458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:24.444588 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:24.446046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:24.447216 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-94d3292b-986b-472f-94af-3337a7e52700 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1600 microversion: 2.20 time: 0.184361 Oct 20 09:05:24.447775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 344/696] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:05:24 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1600 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:05:24.470996 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e6f7932c-e0ac-4279-86e9-dae19a6e062f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:24.471486 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e6f7932c-e0ac-4279-86e9-dae19a6e062f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:24.477230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6f7932c-e0ac-4279-86e9-dae19a6e062f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:24.477702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6f7932c-e0ac-4279-86e9-dae19a6e062f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:24.478129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6f7932c-e0ac-4279-86e9-dae19a6e062f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:24.478833 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6f7932c-e0ac-4279-86e9-dae19a6e062f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:24.479279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6f7932c-e0ac-4279-86e9-dae19a6e062f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:24.480234 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6f7932c-e0ac-4279-86e9-dae19a6e062f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:24.557468 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e6f7932c-e0ac-4279-86e9-dae19a6e062f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.090122 Oct 20 09:05:24.558736 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 345/697] 10.222.0.36 () {60 vars in 1316 bytes} [Thu Oct 20 09:05:24 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/os-interface => generated 285 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 20 09:05:25.019572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:05:25.019985 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:05:25.020444 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:05:25.021050 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:05:25.021650 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:05:25.031234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:25.031641 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:25.032299 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:25.033008 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:25.033836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:25.034658 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:25.053355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Getting quotas for project 020d4019643f4499b5162717fbf7b5b5. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:05:25.057866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Getting quotas for user 14ddd15ab87a4d0fbea1d6dfa910f8ce and project 020d4019643f4499b5162717fbf7b5b5. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:05:25.069145 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:05:25.071621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a3c54c89-6f3a-4e6c-95d5-184dbc52ed18',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:05:25.120872 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b8c7e556-d5ba-4750-bc95-fc4cbd6f9f51 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.792701 Oct 20 09:05:25.121718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 353/698] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:05:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 794 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:05:25.133763 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-30178fb3-2ebf-41db-be8c-bb59fddbcadd tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:25.135113 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-30178fb3-2ebf-41db-be8c-bb59fddbcadd tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:25.225007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-30178fb3-2ebf-41db-be8c-bb59fddbcadd tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:25.233521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-30178fb3-2ebf-41db-be8c-bb59fddbcadd tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1259 microversion: 2.1 time: 0.104744 Oct 20 09:05:25.234207 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 346/699] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:25 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1259 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:25.351213 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-31a1dc23-a06a-4b1e-9f94-c3a937a7fb16 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:25.351983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-31a1dc23-a06a-4b1e-9f94-c3a937a7fb16 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:25.361480 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-31a1dc23-a06a-4b1e-9f94-c3a937a7fb16 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:25.362606 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-31a1dc23-a06a-4b1e-9f94-c3a937a7fb16 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:25.363056 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-31a1dc23-a06a-4b1e-9f94-c3a937a7fb16 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:25.367276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-31a1dc23-a06a-4b1e-9f94-c3a937a7fb16 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:25.369308 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-31a1dc23-a06a-4b1e-9f94-c3a937a7fb16 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:25.370973 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-31a1dc23-a06a-4b1e-9f94-c3a937a7fb16 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:25.522919 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-31a1dc23-a06a-4b1e-9f94-c3a937a7fb16 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:25.533260 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-31a1dc23-a06a-4b1e-9f94-c3a937a7fb16 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1663 microversion: 2.1 time: 0.186508 Oct 20 09:05:25.534329 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 354/700] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:25 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1663 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:05:25.781817 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-30353ef2-f27c-4723-ad57-18bcbdef2196 req-97c220d7-21fb-4e9d-a670-09e7faf40be9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7facb789-628a-44aa-b0ec-7f6aee5617dd", "tag": "f17302af-3ee4-46d2-a6fe-a973b183f4e4"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:25.791937 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-30353ef2-f27c-4723-ad57-18bcbdef2196 req-97c220d7-21fb-4e9d-a670-09e7faf40be9 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:25.792447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-30353ef2-f27c-4723-ad57-18bcbdef2196 req-97c220d7-21fb-4e9d-a670-09e7faf40be9 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:25.792878 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-30353ef2-f27c-4723-ad57-18bcbdef2196 req-97c220d7-21fb-4e9d-a670-09e7faf40be9 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:25.803080 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-30353ef2-f27c-4723-ad57-18bcbdef2196 req-97c220d7-21fb-4e9d-a670-09e7faf40be9 service nova] Creating event network-changed:f17302af-3ee4-46d2-a6fe-a973b183f4e4 for instance 7facb789-628a-44aa-b0ec-7f6aee5617dd on hv-cinder-maste Oct 20 09:05:25.808617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-30353ef2-f27c-4723-ad57-18bcbdef2196 req-97c220d7-21fb-4e9d-a670-09e7faf40be9 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029795 Oct 20 09:05:25.811185 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 347/701] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:05:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:26.256971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6ef13c03-fce7-4e62-aeed-a63356b34ae4 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:26.257938 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6ef13c03-fce7-4e62-aeed-a63356b34ae4 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:26.265677 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ef13c03-fce7-4e62-aeed-a63356b34ae4 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:26.266246 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ef13c03-fce7-4e62-aeed-a63356b34ae4 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:26.266945 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ef13c03-fce7-4e62-aeed-a63356b34ae4 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:26.268093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ef13c03-fce7-4e62-aeed-a63356b34ae4 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:26.268702 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ef13c03-fce7-4e62-aeed-a63356b34ae4 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:26.269203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ef13c03-fce7-4e62-aeed-a63356b34ae4 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:26.371961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6ef13c03-fce7-4e62-aeed-a63356b34ae4 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:26.385068 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6ef13c03-fce7-4e62-aeed-a63356b34ae4 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1255 microversion: 2.1 time: 0.131954 Oct 20 09:05:26.385657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 355/702] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:26 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1255 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:26.558659 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ae70b1b6-77e4-4f6a-b942-42d6997f06ae tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:26.559366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ae70b1b6-77e4-4f6a-b942-42d6997f06ae tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:26.566013 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae70b1b6-77e4-4f6a-b942-42d6997f06ae tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:26.566731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae70b1b6-77e4-4f6a-b942-42d6997f06ae tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:26.567630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae70b1b6-77e4-4f6a-b942-42d6997f06ae tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:26.568661 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae70b1b6-77e4-4f6a-b942-42d6997f06ae tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:26.569273 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae70b1b6-77e4-4f6a-b942-42d6997f06ae tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:26.570073 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae70b1b6-77e4-4f6a-b942-42d6997f06ae tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:26.726663 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ae70b1b6-77e4-4f6a-b942-42d6997f06ae tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:26.739675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ae70b1b6-77e4-4f6a-b942-42d6997f06ae tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1667 microversion: 2.1 time: 0.186227 Oct 20 09:05:26.740241 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 348/703] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:26 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1667 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:27.410092 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-77b1cfc4-8e99-4981-a56b-7d1033e9d629 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:27.410748 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-77b1cfc4-8e99-4981-a56b-7d1033e9d629 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:27.417451 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77b1cfc4-8e99-4981-a56b-7d1033e9d629 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:27.417942 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77b1cfc4-8e99-4981-a56b-7d1033e9d629 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:27.418435 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77b1cfc4-8e99-4981-a56b-7d1033e9d629 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:27.419436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77b1cfc4-8e99-4981-a56b-7d1033e9d629 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:27.419961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77b1cfc4-8e99-4981-a56b-7d1033e9d629 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:27.420303 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77b1cfc4-8e99-4981-a56b-7d1033e9d629 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:27.611836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-77b1cfc4-8e99-4981-a56b-7d1033e9d629 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:27.626866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-77b1cfc4-8e99-4981-a56b-7d1033e9d629 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1317 microversion: 2.1 time: 0.221228 Oct 20 09:05:27.627700 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 356/704] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:27 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1317 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:28.653088 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-50e98f57-49bd-461c-8ba3-18cf5124a69d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:28.653642 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-50e98f57-49bd-461c-8ba3-18cf5124a69d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:28.660273 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50e98f57-49bd-461c-8ba3-18cf5124a69d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:28.660972 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50e98f57-49bd-461c-8ba3-18cf5124a69d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:28.661409 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50e98f57-49bd-461c-8ba3-18cf5124a69d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:28.662121 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50e98f57-49bd-461c-8ba3-18cf5124a69d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:28.662575 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50e98f57-49bd-461c-8ba3-18cf5124a69d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:28.662993 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50e98f57-49bd-461c-8ba3-18cf5124a69d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:28.977004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-50e98f57-49bd-461c-8ba3-18cf5124a69d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:28.984839 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-50e98f57-49bd-461c-8ba3-18cf5124a69d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.336299 Oct 20 09:05:28.985408 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 349/705] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:28 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:30.008039 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5b64ccf0-70d0-4bff-aa43-653bc97fdc6c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:30.008609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5b64ccf0-70d0-4bff-aa43-653bc97fdc6c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:30.015393 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b64ccf0-70d0-4bff-aa43-653bc97fdc6c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:30.015872 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b64ccf0-70d0-4bff-aa43-653bc97fdc6c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:30.016411 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b64ccf0-70d0-4bff-aa43-653bc97fdc6c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:30.017179 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b64ccf0-70d0-4bff-aa43-653bc97fdc6c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:30.017660 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b64ccf0-70d0-4bff-aa43-653bc97fdc6c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:30.018088 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b64ccf0-70d0-4bff-aa43-653bc97fdc6c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:30.223772 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5b64ccf0-70d0-4bff-aa43-653bc97fdc6c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:30.235823 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5b64ccf0-70d0-4bff-aa43-653bc97fdc6c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.231935 Oct 20 09:05:30.236407 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 357/706] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:30 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:31.259302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2abe4871-e5e7-4ccc-a817-938fd3ccc610 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:31.260081 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2abe4871-e5e7-4ccc-a817-938fd3ccc610 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:31.262257 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=79,nova_api:UPDATE=1,nova_api:INSERT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:31.267551 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2abe4871-e5e7-4ccc-a817-938fd3ccc610 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:31.268028 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2abe4871-e5e7-4ccc-a817-938fd3ccc610 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:31.268463 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2abe4871-e5e7-4ccc-a817-938fd3ccc610 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:31.269153 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2abe4871-e5e7-4ccc-a817-938fd3ccc610 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:31.269623 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2abe4871-e5e7-4ccc-a817-938fd3ccc610 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:31.270176 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2abe4871-e5e7-4ccc-a817-938fd3ccc610 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:31.407326 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2abe4871-e5e7-4ccc-a817-938fd3ccc610 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:31.415500 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2abe4871-e5e7-4ccc-a817-938fd3ccc610 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.159991 Oct 20 09:05:31.416301 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 350/707] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:31 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:32.439408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-41d850e7-94fc-4a79-8227-a5f5f908f802 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:32.440034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-41d850e7-94fc-4a79-8227-a5f5f908f802 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:32.446802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-41d850e7-94fc-4a79-8227-a5f5f908f802 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:32.447295 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-41d850e7-94fc-4a79-8227-a5f5f908f802 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:32.447726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-41d850e7-94fc-4a79-8227-a5f5f908f802 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:32.449022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-41d850e7-94fc-4a79-8227-a5f5f908f802 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:32.449454 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-41d850e7-94fc-4a79-8227-a5f5f908f802 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:32.449883 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-41d850e7-94fc-4a79-8227-a5f5f908f802 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:32.591295 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-41d850e7-94fc-4a79-8227-a5f5f908f802 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:32.598932 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-41d850e7-94fc-4a79-8227-a5f5f908f802 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.163334 Oct 20 09:05:32.599500 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 358/708] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:32 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:33.302063 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5bb09ed0-9179-4132-85b9-797c45f98006 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:33.302690 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5bb09ed0-9179-4132-85b9-797c45f98006 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:33.309516 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bb09ed0-9179-4132-85b9-797c45f98006 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:33.309999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bb09ed0-9179-4132-85b9-797c45f98006 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:33.310437 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bb09ed0-9179-4132-85b9-797c45f98006 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:33.311161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bb09ed0-9179-4132-85b9-797c45f98006 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:33.311611 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bb09ed0-9179-4132-85b9-797c45f98006 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:33.312035 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5bb09ed0-9179-4132-85b9-797c45f98006 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:33.397801 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-5bb09ed0-9179-4132-85b9-797c45f98006 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lazy-loading 'services' on Instance uuid ea0966d3-1311-4690-a69f-d3df935a76e1 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:05:33.471770 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5bb09ed0-9179-4132-85b9-797c45f98006 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "DELETE /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments/408e1832-d211-409c-a48e-dd0ec66831c0" status: 202 len: 0 microversion: 2.1 time: 0.173671 Oct 20 09:05:33.472690 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 351/709] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:05:33 2022] DELETE /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments/408e1832-d211-409c-a48e-dd0ec66831c0 => generated 0 bytes in 175 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:05:33.481817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f560529f-463b-45e7-a0ed-98360e2eafad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:33.487335 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f560529f-463b-45e7-a0ed-98360e2eafad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:33.491495 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d51db3d4-05f7-4afc-bed1-107182e69634 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:33.492080 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d51db3d4-05f7-4afc-bed1-107182e69634 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:33.494700 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f560529f-463b-45e7-a0ed-98360e2eafad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:33.495542 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f560529f-463b-45e7-a0ed-98360e2eafad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:33.496274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f560529f-463b-45e7-a0ed-98360e2eafad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:33.497889 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f560529f-463b-45e7-a0ed-98360e2eafad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:33.498605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f560529f-463b-45e7-a0ed-98360e2eafad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:33.499233 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51db3d4-05f7-4afc-bed1-107182e69634 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:33.499488 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f560529f-463b-45e7-a0ed-98360e2eafad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:33.499727 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51db3d4-05f7-4afc-bed1-107182e69634 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:33.500248 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51db3d4-05f7-4afc-bed1-107182e69634 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:33.501096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51db3d4-05f7-4afc-bed1-107182e69634 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:33.501557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51db3d4-05f7-4afc-bed1-107182e69634 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:33.502131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d51db3d4-05f7-4afc-bed1-107182e69634 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:33.536042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f560529f-463b-45e7-a0ed-98360e2eafad tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058867 Oct 20 09:05:33.537431 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 359/710] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:05:33 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:33.549940 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d51db3d4-05f7-4afc-bed1-107182e69634 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/ips" status: 200 len: 149 microversion: 2.1 time: 0.062639 Oct 20 09:05:33.550548 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 352/711] 10.222.0.36 () {58 vars in 1248 bytes} [Thu Oct 20 09:05:33 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/ips => generated 149 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:05:33.622319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-96ee5f29-1095-4c39-a536-38c174d20424 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:33.623019 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-96ee5f29-1095-4c39-a536-38c174d20424 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:33.629311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-96ee5f29-1095-4c39-a536-38c174d20424 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:33.629794 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-96ee5f29-1095-4c39-a536-38c174d20424 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:33.630226 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-96ee5f29-1095-4c39-a536-38c174d20424 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:33.630975 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-96ee5f29-1095-4c39-a536-38c174d20424 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:33.631424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-96ee5f29-1095-4c39-a536-38c174d20424 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:33.631842 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-96ee5f29-1095-4c39-a536-38c174d20424 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:33.792758 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-96ee5f29-1095-4c39-a536-38c174d20424 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:33.803123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-96ee5f29-1095-4c39-a536-38c174d20424 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.184613 Oct 20 09:05:33.803731 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 360/712] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:33 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:34.549329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1c63bc16-6c7c-4bfd-9354-81c2ae8e70e8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:34.552455 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1c63bc16-6c7c-4bfd-9354-81c2ae8e70e8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:34.559764 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c63bc16-6c7c-4bfd-9354-81c2ae8e70e8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:34.560256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c63bc16-6c7c-4bfd-9354-81c2ae8e70e8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:34.560697 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c63bc16-6c7c-4bfd-9354-81c2ae8e70e8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:34.561405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c63bc16-6c7c-4bfd-9354-81c2ae8e70e8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:34.561908 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c63bc16-6c7c-4bfd-9354-81c2ae8e70e8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:34.562287 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1c63bc16-6c7c-4bfd-9354-81c2ae8e70e8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:34.595339 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1c63bc16-6c7c-4bfd-9354-81c2ae8e70e8 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049843 Oct 20 09:05:34.595915 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 353/713] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:05:34 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:34.827743 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d8705025-d3c5-4a46-9b0f-d5024351a50c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:34.828282 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d8705025-d3c5-4a46-9b0f-d5024351a50c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:34.835035 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8705025-d3c5-4a46-9b0f-d5024351a50c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:34.835518 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8705025-d3c5-4a46-9b0f-d5024351a50c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:34.835954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8705025-d3c5-4a46-9b0f-d5024351a50c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:34.836716 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8705025-d3c5-4a46-9b0f-d5024351a50c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:34.837357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8705025-d3c5-4a46-9b0f-d5024351a50c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:34.837859 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8705025-d3c5-4a46-9b0f-d5024351a50c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:34.984017 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-953cd1eb-b90d-474a-bc5c-cb644512cfbd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:34.984851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d8705025-d3c5-4a46-9b0f-d5024351a50c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:34.986504 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-953cd1eb-b90d-474a-bc5c-cb644512cfbd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:34.992393 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-953cd1eb-b90d-474a-bc5c-cb644512cfbd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:34.993092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-953cd1eb-b90d-474a-bc5c-cb644512cfbd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:34.993534 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-953cd1eb-b90d-474a-bc5c-cb644512cfbd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:34.994500 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-953cd1eb-b90d-474a-bc5c-cb644512cfbd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:34.995255 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-953cd1eb-b90d-474a-bc5c-cb644512cfbd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:34.995689 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-953cd1eb-b90d-474a-bc5c-cb644512cfbd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:34.995942 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d8705025-d3c5-4a46-9b0f-d5024351a50c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.172476 Oct 20 09:05:34.996593 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 361/714] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:34 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:35.233321 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=6 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:35.610349 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bd85adc2-a49d-4223-9a7f-df12c3f4d779 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:35.610759 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bd85adc2-a49d-4223-9a7f-df12c3f4d779 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:35.612385 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:35.613035 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=6 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:35.618600 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd85adc2-a49d-4223-9a7f-df12c3f4d779 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:35.619081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd85adc2-a49d-4223-9a7f-df12c3f4d779 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:35.619508 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd85adc2-a49d-4223-9a7f-df12c3f4d779 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:35.620831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd85adc2-a49d-4223-9a7f-df12c3f4d779 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:35.621278 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd85adc2-a49d-4223-9a7f-df12c3f4d779 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:35.621697 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd85adc2-a49d-4223-9a7f-df12c3f4d779 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:35.662496 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bd85adc2-a49d-4223-9a7f-df12c3f4d779 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058640 Oct 20 09:05:35.663586 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 362/715] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:05:35 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:05:36.020300 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-76164dbf-03e5-4a2f-8619-d07d995330d5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:36.021049 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-76164dbf-03e5-4a2f-8619-d07d995330d5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:36.028044 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76164dbf-03e5-4a2f-8619-d07d995330d5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:36.028591 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76164dbf-03e5-4a2f-8619-d07d995330d5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:36.029026 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76164dbf-03e5-4a2f-8619-d07d995330d5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:36.029740 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76164dbf-03e5-4a2f-8619-d07d995330d5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:36.030182 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76164dbf-03e5-4a2f-8619-d07d995330d5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:36.030635 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-76164dbf-03e5-4a2f-8619-d07d995330d5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:36.160420 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-76164dbf-03e5-4a2f-8619-d07d995330d5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:36.172269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-76164dbf-03e5-4a2f-8619-d07d995330d5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.155753 Oct 20 09:05:36.173474 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 363/716] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:36 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:05:36.676106 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3466a8dd-9175-4ece-886f-4e4e334d3034 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:36.678703 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3466a8dd-9175-4ece-886f-4e4e334d3034 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:36.686071 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3466a8dd-9175-4ece-886f-4e4e334d3034 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:36.686616 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3466a8dd-9175-4ece-886f-4e4e334d3034 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:36.687053 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3466a8dd-9175-4ece-886f-4e4e334d3034 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:36.687831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3466a8dd-9175-4ece-886f-4e4e334d3034 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:36.688446 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3466a8dd-9175-4ece-886f-4e4e334d3034 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:36.688924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3466a8dd-9175-4ece-886f-4e4e334d3034 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:36.706382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=100 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:36.725691 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3466a8dd-9175-4ece-886f-4e4e334d3034 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052724 Oct 20 09:05:36.726380 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 364/717] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:05:36 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:05:36.894564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-953cd1eb-b90d-474a-bc5c-cb644512cfbd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.914923 Oct 20 09:05:36.895161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 354/718] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:05:34 2022] POST /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments => generated 194 bytes in 1916 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:05:37.195893 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0d64d218-aa87-42f9-896d-ce82b9ff90a3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:37.196612 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0d64d218-aa87-42f9-896d-ce82b9ff90a3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:37.203584 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d64d218-aa87-42f9-896d-ce82b9ff90a3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:37.204053 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d64d218-aa87-42f9-896d-ce82b9ff90a3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:37.204380 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d64d218-aa87-42f9-896d-ce82b9ff90a3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:37.205039 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d64d218-aa87-42f9-896d-ce82b9ff90a3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:37.205482 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d64d218-aa87-42f9-896d-ce82b9ff90a3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:37.205899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d64d218-aa87-42f9-896d-ce82b9ff90a3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:37.357955 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-219b6719-4e93-41e0-94ab-2222af0ae8fe req-f6655b5b-ab5a-4fd9-934d-64ea15e460c9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03c3c04f-2850-480f-8d15-6f15327734e0", "tag": "3c5f5de7-af1f-4408-ba7d-6be5f0344198"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:37.368105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-219b6719-4e93-41e0-94ab-2222af0ae8fe req-f6655b5b-ab5a-4fd9-934d-64ea15e460c9 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:37.369109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-219b6719-4e93-41e0-94ab-2222af0ae8fe req-f6655b5b-ab5a-4fd9-934d-64ea15e460c9 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:37.369544 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-219b6719-4e93-41e0-94ab-2222af0ae8fe req-f6655b5b-ab5a-4fd9-934d-64ea15e460c9 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:37.381512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-219b6719-4e93-41e0-94ab-2222af0ae8fe req-f6655b5b-ab5a-4fd9-934d-64ea15e460c9 service nova] Creating event network-changed:3c5f5de7-af1f-4408-ba7d-6be5f0344198 for instance 03c3c04f-2850-480f-8d15-6f15327734e0 on hv-cinder-maste Oct 20 09:05:37.387624 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-219b6719-4e93-41e0-94ab-2222af0ae8fe req-f6655b5b-ab5a-4fd9-934d-64ea15e460c9 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033922 Oct 20 09:05:37.388237 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 355/719] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:05:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:05:37.423076 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0d64d218-aa87-42f9-896d-ce82b9ff90a3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:37.431642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0d64d218-aa87-42f9-896d-ce82b9ff90a3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.239671 Oct 20 09:05:37.433162 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 365/720] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:37 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:37.740421 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2f8f8b98-9ab4-49ce-9d30-ba51f2ba6d6d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:37.742849 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2f8f8b98-9ab4-49ce-9d30-ba51f2ba6d6d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:37.753152 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2f8f8b98-9ab4-49ce-9d30-ba51f2ba6d6d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:37.754018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2f8f8b98-9ab4-49ce-9d30-ba51f2ba6d6d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:37.754661 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2f8f8b98-9ab4-49ce-9d30-ba51f2ba6d6d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:37.756124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2f8f8b98-9ab4-49ce-9d30-ba51f2ba6d6d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:37.756595 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2f8f8b98-9ab4-49ce-9d30-ba51f2ba6d6d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:37.757030 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2f8f8b98-9ab4-49ce-9d30-ba51f2ba6d6d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:37.788543 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2f8f8b98-9ab4-49ce-9d30-ba51f2ba6d6d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052982 Oct 20 09:05:37.789532 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 356/721] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:05:37 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:38.454745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-267e2ab9-fc3f-4d17-abee-707f0c1566f2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:38.455342 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-267e2ab9-fc3f-4d17-abee-707f0c1566f2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:38.461667 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-267e2ab9-fc3f-4d17-abee-707f0c1566f2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:38.462134 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-267e2ab9-fc3f-4d17-abee-707f0c1566f2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:38.462590 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-267e2ab9-fc3f-4d17-abee-707f0c1566f2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:38.463269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-267e2ab9-fc3f-4d17-abee-707f0c1566f2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:38.464836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-267e2ab9-fc3f-4d17-abee-707f0c1566f2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:38.465522 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-267e2ab9-fc3f-4d17-abee-707f0c1566f2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:38.594031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-267e2ab9-fc3f-4d17-abee-707f0c1566f2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:38.601580 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-267e2ab9-fc3f-4d17-abee-707f0c1566f2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.150727 Oct 20 09:05:38.602147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 366/722] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:38 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:38.801513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bf03030c-ec9d-4497-8dc2-0c2720831628 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:38.803995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bf03030c-ec9d-4497-8dc2-0c2720831628 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:38.810296 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf03030c-ec9d-4497-8dc2-0c2720831628 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:38.810807 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf03030c-ec9d-4497-8dc2-0c2720831628 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:38.811241 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf03030c-ec9d-4497-8dc2-0c2720831628 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:38.811913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf03030c-ec9d-4497-8dc2-0c2720831628 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:38.812361 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf03030c-ec9d-4497-8dc2-0c2720831628 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:38.812780 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bf03030c-ec9d-4497-8dc2-0c2720831628 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:38.843242 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bf03030c-ec9d-4497-8dc2-0c2720831628 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045999 Oct 20 09:05:38.843793 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 357/723] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:05:38 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 20 09:05:38.925139 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ac3cda08-4e13-4535-99b4-c243334a820f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:38.925633 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ac3cda08-4e13-4535-99b4-c243334a820f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:38.931514 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac3cda08-4e13-4535-99b4-c243334a820f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:38.931996 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac3cda08-4e13-4535-99b4-c243334a820f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:38.932931 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac3cda08-4e13-4535-99b4-c243334a820f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:38.933623 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac3cda08-4e13-4535-99b4-c243334a820f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:38.934075 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac3cda08-4e13-4535-99b4-c243334a820f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:38.934522 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac3cda08-4e13-4535-99b4-c243334a820f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:38.959475 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ac3cda08-4e13-4535-99b4-c243334a820f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:05:39.035002 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ac3cda08-4e13-4535-99b4-c243334a820f tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "DELETE /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 204 len: 0 microversion: 2.1 time: 0.113438 Oct 20 09:05:39.035582 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 367/724] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:05:38 2022] DELETE /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:05:39.043508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7ee23197-d6ae-4bf9-a9d1-f995b1104a9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:39.044328 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7ee23197-d6ae-4bf9-a9d1-f995b1104a9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:39.050163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee23197-d6ae-4bf9-a9d1-f995b1104a9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:39.050854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee23197-d6ae-4bf9-a9d1-f995b1104a9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:39.051290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee23197-d6ae-4bf9-a9d1-f995b1104a9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:39.052024 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee23197-d6ae-4bf9-a9d1-f995b1104a9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:39.052415 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee23197-d6ae-4bf9-a9d1-f995b1104a9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:39.052833 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee23197-d6ae-4bf9-a9d1-f995b1104a9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:39.180151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7ee23197-d6ae-4bf9-a9d1-f995b1104a9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:39.188274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7ee23197-d6ae-4bf9-a9d1-f995b1104a9c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1687 microversion: 2.1 time: 0.148506 Oct 20 09:05:39.188838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 358/725] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:39 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1687 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:39.625510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-760f45b0-8789-475f-ac40-6adb833da4d8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:39.626202 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-760f45b0-8789-475f-ac40-6adb833da4d8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:39.632627 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-760f45b0-8789-475f-ac40-6adb833da4d8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:39.633048 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-760f45b0-8789-475f-ac40-6adb833da4d8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:39.633466 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-760f45b0-8789-475f-ac40-6adb833da4d8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:39.634150 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-760f45b0-8789-475f-ac40-6adb833da4d8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:39.634617 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-760f45b0-8789-475f-ac40-6adb833da4d8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:39.635037 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-760f45b0-8789-475f-ac40-6adb833da4d8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:39.760483 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-760f45b0-8789-475f-ac40-6adb833da4d8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:39.768053 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-760f45b0-8789-475f-ac40-6adb833da4d8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.146732 Oct 20 09:05:39.768622 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 368/726] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:39 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:40.212033 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0a2f4d1f-fba6-41cf-8e80-1ddc5799b611 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:40.212652 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0a2f4d1f-fba6-41cf-8e80-1ddc5799b611 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:40.220143 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2f4d1f-fba6-41cf-8e80-1ddc5799b611 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:40.220617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2f4d1f-fba6-41cf-8e80-1ddc5799b611 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:40.221051 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2f4d1f-fba6-41cf-8e80-1ddc5799b611 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:40.221726 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2f4d1f-fba6-41cf-8e80-1ddc5799b611 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:40.222169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2f4d1f-fba6-41cf-8e80-1ddc5799b611 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:40.222616 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2f4d1f-fba6-41cf-8e80-1ddc5799b611 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:40.352809 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0a2f4d1f-fba6-41cf-8e80-1ddc5799b611 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:40.360695 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0a2f4d1f-fba6-41cf-8e80-1ddc5799b611 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1687 microversion: 2.1 time: 0.152722 Oct 20 09:05:40.361254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 359/727] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:40 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1687 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:40.791332 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-06eca8fb-b8f6-4549-94fa-a6abfe4b9418 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:40.792594 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-06eca8fb-b8f6-4549-94fa-a6abfe4b9418 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:40.799065 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-06eca8fb-b8f6-4549-94fa-a6abfe4b9418 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:40.799545 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-06eca8fb-b8f6-4549-94fa-a6abfe4b9418 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:40.800010 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-06eca8fb-b8f6-4549-94fa-a6abfe4b9418 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:40.800693 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-06eca8fb-b8f6-4549-94fa-a6abfe4b9418 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:40.801144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-06eca8fb-b8f6-4549-94fa-a6abfe4b9418 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:40.801575 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-06eca8fb-b8f6-4549-94fa-a6abfe4b9418 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:40.925006 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-06eca8fb-b8f6-4549-94fa-a6abfe4b9418 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:40.932618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-06eca8fb-b8f6-4549-94fa-a6abfe4b9418 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.145112 Oct 20 09:05:40.933180 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 369/728] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:40 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:41.490127 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1225720c-5d56-42c2-a2da-f3990037b6cf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:41.490762 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1225720c-5d56-42c2-a2da-f3990037b6cf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:41.497216 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1225720c-5d56-42c2-a2da-f3990037b6cf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:41.497695 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1225720c-5d56-42c2-a2da-f3990037b6cf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:41.498138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1225720c-5d56-42c2-a2da-f3990037b6cf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:41.498854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1225720c-5d56-42c2-a2da-f3990037b6cf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:41.499305 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1225720c-5d56-42c2-a2da-f3990037b6cf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:41.499728 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1225720c-5d56-42c2-a2da-f3990037b6cf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:41.627421 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1225720c-5d56-42c2-a2da-f3990037b6cf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:41.635403 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1225720c-5d56-42c2-a2da-f3990037b6cf tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1687 microversion: 2.1 time: 0.255689 Oct 20 09:05:41.635944 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 360/729] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:41 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1687 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:41.955134 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d20707ca-9925-4e51-b9f7-620fce2503d1 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:41.955671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d20707ca-9925-4e51-b9f7-620fce2503d1 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:41.962054 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d20707ca-9925-4e51-b9f7-620fce2503d1 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:41.962565 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d20707ca-9925-4e51-b9f7-620fce2503d1 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:41.963001 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d20707ca-9925-4e51-b9f7-620fce2503d1 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:41.964215 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d20707ca-9925-4e51-b9f7-620fce2503d1 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:41.964664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d20707ca-9925-4e51-b9f7-620fce2503d1 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:41.965088 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d20707ca-9925-4e51-b9f7-620fce2503d1 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:42.097240 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d20707ca-9925-4e51-b9f7-620fce2503d1 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:42.105993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d20707ca-9925-4e51-b9f7-620fce2503d1 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.154737 Oct 20 09:05:42.106694 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 370/730] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:41 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:42.658890 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-009f0b4f-c859-498b-95c5-3e354a2d6e31 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:42.659537 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-009f0b4f-c859-498b-95c5-3e354a2d6e31 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:42.665815 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-009f0b4f-c859-498b-95c5-3e354a2d6e31 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:42.666295 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-009f0b4f-c859-498b-95c5-3e354a2d6e31 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:42.667293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-009f0b4f-c859-498b-95c5-3e354a2d6e31 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:42.667973 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-009f0b4f-c859-498b-95c5-3e354a2d6e31 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:42.668420 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-009f0b4f-c859-498b-95c5-3e354a2d6e31 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:42.668845 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-009f0b4f-c859-498b-95c5-3e354a2d6e31 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:42.800426 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-009f0b4f-c859-498b-95c5-3e354a2d6e31 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:42.802799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:42.810009 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-009f0b4f-c859-498b-95c5-3e354a2d6e31 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1687 microversion: 2.1 time: 0.155001 Oct 20 09:05:42.810656 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 361/731] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:42 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1687 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:05:43.129002 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c886cdb3-4d43-4ad1-8bcd-380ecacff48d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:43.129605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c886cdb3-4d43-4ad1-8bcd-380ecacff48d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:43.136578 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c886cdb3-4d43-4ad1-8bcd-380ecacff48d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:43.137052 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c886cdb3-4d43-4ad1-8bcd-380ecacff48d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:43.137478 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c886cdb3-4d43-4ad1-8bcd-380ecacff48d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:43.138158 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c886cdb3-4d43-4ad1-8bcd-380ecacff48d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:43.138624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c886cdb3-4d43-4ad1-8bcd-380ecacff48d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:43.139040 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c886cdb3-4d43-4ad1-8bcd-380ecacff48d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:43.264361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c886cdb3-4d43-4ad1-8bcd-380ecacff48d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:43.271832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c886cdb3-4d43-4ad1-8bcd-380ecacff48d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.146659 Oct 20 09:05:43.272395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 371/732] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:43 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:43.516507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-78d52a96-7d73-403b-bbc2-4ef85298788a req-16307f5a-6502-4bf0-816e-d316cd381f8a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "03c3c04f-2850-480f-8d15-6f15327734e0", "name": "network-vif-plugged", "status": "completed", "tag": "3c5f5de7-af1f-4408-ba7d-6be5f0344198"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:43.528324 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-78d52a96-7d73-403b-bbc2-4ef85298788a req-16307f5a-6502-4bf0-816e-d316cd381f8a service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:43.528885 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-78d52a96-7d73-403b-bbc2-4ef85298788a req-16307f5a-6502-4bf0-816e-d316cd381f8a service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:43.529323 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-78d52a96-7d73-403b-bbc2-4ef85298788a req-16307f5a-6502-4bf0-816e-d316cd381f8a service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:43.540314 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-78d52a96-7d73-403b-bbc2-4ef85298788a req-16307f5a-6502-4bf0-816e-d316cd381f8a service nova] Creating event network-vif-plugged:3c5f5de7-af1f-4408-ba7d-6be5f0344198 for instance 03c3c04f-2850-480f-8d15-6f15327734e0 on hv-cinder-maste Oct 20 09:05:43.545034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-78d52a96-7d73-403b-bbc2-4ef85298788a req-16307f5a-6502-4bf0-816e-d316cd381f8a service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032561 Oct 20 09:05:43.545620 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 362/733] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:05:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:43.832592 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-38a7e873-d8fa-410a-966f-27d6a171e588 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:43.833187 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-38a7e873-d8fa-410a-966f-27d6a171e588 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:43.839237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38a7e873-d8fa-410a-966f-27d6a171e588 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:43.839713 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38a7e873-d8fa-410a-966f-27d6a171e588 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:43.840404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38a7e873-d8fa-410a-966f-27d6a171e588 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:43.841090 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38a7e873-d8fa-410a-966f-27d6a171e588 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:43.841529 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38a7e873-d8fa-410a-966f-27d6a171e588 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:43.842039 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-38a7e873-d8fa-410a-966f-27d6a171e588 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:44.112817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-38a7e873-d8fa-410a-966f-27d6a171e588 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:44.122670 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-38a7e873-d8fa-410a-966f-27d6a171e588 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1687 microversion: 2.1 time: 0.293901 Oct 20 09:05:44.123954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 372/734] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:43 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1687 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:44.297124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-dfb84446-9ec8-4906-bffe-54d599689cc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:44.297922 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-dfb84446-9ec8-4906-bffe-54d599689cc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:44.304101 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dfb84446-9ec8-4906-bffe-54d599689cc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:44.304587 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dfb84446-9ec8-4906-bffe-54d599689cc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:44.305021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dfb84446-9ec8-4906-bffe-54d599689cc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:44.305715 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dfb84446-9ec8-4906-bffe-54d599689cc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:44.306163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dfb84446-9ec8-4906-bffe-54d599689cc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:44.306615 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dfb84446-9ec8-4906-bffe-54d599689cc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:44.447888 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:44.450989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:44.458286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:44.459106 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:44.459775 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:44.460733 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:44.461401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:44.462036 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:44.467658 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-dfb84446-9ec8-4906-bffe-54d599689cc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:44.476109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-dfb84446-9ec8-4906-bffe-54d599689cc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1359 microversion: 2.1 time: 0.184806 Oct 20 09:05:44.477159 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 363/735] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:44 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1359 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:44.557745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Invalid volume: volume 2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2 is already attached to instances: 06d8b443-0b7d-4287-8469-eb4a35863289 Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: POST /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments HTTP/1.1 Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 74 Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:05:44.559506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {"volumeAttachment": {"volumeId": "2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2"}} Oct 20 09:05:44.560884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Invalid volume: volume 2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2 is already attached to instances: 06d8b443-0b7d-4287-8469-eb4a35863289 Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'server_id': '06d8b443-0b7d-4287-8469-eb4a35863289', 'body': {'volumeAttachment': {'volumeId': '2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2'}}} Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2 is already attached to instances: 06d8b443-0b7d-4287-8469-eb4a35863289 Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:44.561631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2 is already attached to instances: 06d8b443-0b7d-4287-8469-eb4a35863289 Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:44.562846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:05:44.564081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:05:44.564081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:05:44.564081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:05:44.564081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2 is already attached to instances: 06d8b443-0b7d-4287-8469-eb4a35863289 Oct 20 09:05:44.564081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:44.564081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 400 to user: Invalid volume: volume 2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2 is already attached to instances: 06d8b443-0b7d-4287-8469-eb4a35863289 {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:05:44.564081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-47ff9cb1-f5fb-4a6c-be0a-973e2d2c4cbe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.117279 Oct 20 09:05:44.564081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 373/736] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:05:44 2022] POST /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments => generated 174 bytes in 120 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 20 09:05:44.634601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c6c74bb6-8c44-46dd-95a3-71dcc97283ff tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:44.635178 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c6c74bb6-8c44-46dd-95a3-71dcc97283ff tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:44.641620 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6c74bb6-8c44-46dd-95a3-71dcc97283ff tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:44.642106 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6c74bb6-8c44-46dd-95a3-71dcc97283ff tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:44.642578 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6c74bb6-8c44-46dd-95a3-71dcc97283ff tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:44.643275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6c74bb6-8c44-46dd-95a3-71dcc97283ff tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:44.643723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6c74bb6-8c44-46dd-95a3-71dcc97283ff tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:44.644147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6c74bb6-8c44-46dd-95a3-71dcc97283ff tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:44.732006 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-c6c74bb6-8c44-46dd-95a3-71dcc97283ff tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lazy-loading 'services' on Instance uuid 06d8b443-0b7d-4287-8469-eb4a35863289 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:05:44.771121 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-df031cd4-2eef-48ed-a723-b3204598c788 req-34b58659-25f5-4f8c-8466-24bc2a46038d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ea0966d3-1311-4690-a69f-d3df935a76e1", "name": "network-vif-deleted", "tag": "b9bd9664-f1f8-415b-84b5-9c0f08f3fc00"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:44.781009 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-df031cd4-2eef-48ed-a723-b3204598c788 req-34b58659-25f5-4f8c-8466-24bc2a46038d service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:44.781747 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-df031cd4-2eef-48ed-a723-b3204598c788 req-34b58659-25f5-4f8c-8466-24bc2a46038d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:44.782685 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-df031cd4-2eef-48ed-a723-b3204598c788 req-34b58659-25f5-4f8c-8466-24bc2a46038d service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:44.795620 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-df031cd4-2eef-48ed-a723-b3204598c788 req-34b58659-25f5-4f8c-8466-24bc2a46038d service nova] Creating event network-vif-deleted:b9bd9664-f1f8-415b-84b5-9c0f08f3fc00 for instance ea0966d3-1311-4690-a69f-d3df935a76e1 on hv-cinder-maste Oct 20 09:05:44.801877 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-df031cd4-2eef-48ed-a723-b3204598c788 req-34b58659-25f5-4f8c-8466-24bc2a46038d service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034398 Oct 20 09:05:44.802850 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 374/737] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:05:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:05:44.817801 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c6c74bb6-8c44-46dd-95a3-71dcc97283ff tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments/2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2" status: 202 len: 0 microversion: 2.1 time: 0.186734 Oct 20 09:05:44.822410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 364/738] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:05:44 2022] DELETE /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments/2c423cb4-f982-4f0d-ab62-8ed07a6fcfb2 => generated 0 bytes in 192 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:05:44.828393 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5c09e9ff-9115-4f18-b3d1-7666283857f0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:44.830827 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5c09e9ff-9115-4f18-b3d1-7666283857f0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:44.837948 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c09e9ff-9115-4f18-b3d1-7666283857f0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:44.838510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c09e9ff-9115-4f18-b3d1-7666283857f0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:44.838944 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c09e9ff-9115-4f18-b3d1-7666283857f0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:44.839909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c09e9ff-9115-4f18-b3d1-7666283857f0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:44.840773 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c09e9ff-9115-4f18-b3d1-7666283857f0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:44.841494 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5c09e9ff-9115-4f18-b3d1-7666283857f0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:44.876393 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5c09e9ff-9115-4f18-b3d1-7666283857f0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052488 Oct 20 09:05:44.877047 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 375/739] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:05:44 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:05:45.147194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-85682b40-2992-497e-a667-a76d2ee007ec tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:45.147895 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-85682b40-2992-497e-a667-a76d2ee007ec tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:45.155396 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85682b40-2992-497e-a667-a76d2ee007ec tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:45.155897 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85682b40-2992-497e-a667-a76d2ee007ec tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:45.156351 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85682b40-2992-497e-a667-a76d2ee007ec tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:45.157052 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85682b40-2992-497e-a667-a76d2ee007ec tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:45.157704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85682b40-2992-497e-a667-a76d2ee007ec tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:45.158134 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85682b40-2992-497e-a667-a76d2ee007ec tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:45.237503 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-85682b40-2992-497e-a667-a76d2ee007ec tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:45.246402 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-85682b40-2992-497e-a667-a76d2ee007ec tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1330 microversion: 2.1 time: 0.103558 Oct 20 09:05:45.247006 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 365/740] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:45 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1330 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:45.499414 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-df813bf1-4a96-4d68-a247-b6e551cded65 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:45.500024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-df813bf1-4a96-4d68-a247-b6e551cded65 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:45.506494 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-df813bf1-4a96-4d68-a247-b6e551cded65 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:45.506997 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-df813bf1-4a96-4d68-a247-b6e551cded65 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:45.507431 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-df813bf1-4a96-4d68-a247-b6e551cded65 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:45.508125 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-df813bf1-4a96-4d68-a247-b6e551cded65 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:45.508573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-df813bf1-4a96-4d68-a247-b6e551cded65 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:45.508993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-df813bf1-4a96-4d68-a247-b6e551cded65 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:45.641504 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-df813bf1-4a96-4d68-a247-b6e551cded65 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:45.649612 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-df813bf1-4a96-4d68-a247-b6e551cded65 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1549 microversion: 2.1 time: 0.154028 Oct 20 09:05:45.650163 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 376/741] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:45 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1549 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:45.672656 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4bd33691-8cd7-4d85-ace5-0512af2b3b12 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:45.673355 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4bd33691-8cd7-4d85-ace5-0512af2b3b12 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:45.680061 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bd33691-8cd7-4d85-ace5-0512af2b3b12 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:45.680551 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bd33691-8cd7-4d85-ace5-0512af2b3b12 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:45.680988 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bd33691-8cd7-4d85-ace5-0512af2b3b12 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:45.681696 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bd33691-8cd7-4d85-ace5-0512af2b3b12 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:45.682703 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bd33691-8cd7-4d85-ace5-0512af2b3b12 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:45.683182 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4bd33691-8cd7-4d85-ace5-0512af2b3b12 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:45.833921 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4bd33691-8cd7-4d85-ace5-0512af2b3b12 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:45.842417 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4bd33691-8cd7-4d85-ace5-0512af2b3b12 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1549 microversion: 2.1 time: 0.173687 Oct 20 09:05:45.843018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 366/742] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:45 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1549 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:45.863984 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:45.868680 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:45.869330 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Searching by: {'deleted': False, 'project_id': '020d4019643f4499b5162717fbf7b5b5'} {{(pid=119067) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 20 09:05:45.872042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:45.878047 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:45.878550 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:45.878978 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:45.879683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:45.880129 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:45.880542 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:45.977660 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.multi_cell_list [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=119067) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 20 09:05:46.004436 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2b00cb5e-b840-4b4f-80be-d782cd7768a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:46.006803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2b00cb5e-b840-4b4f-80be-d782cd7768a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:46.012759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b00cb5e-b840-4b4f-80be-d782cd7768a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:46.013247 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b00cb5e-b840-4b4f-80be-d782cd7768a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:46.013688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b00cb5e-b840-4b4f-80be-d782cd7768a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:46.014382 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b00cb5e-b840-4b4f-80be-d782cd7768a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:46.014867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b00cb5e-b840-4b4f-80be-d782cd7768a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:46.015294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2b00cb5e-b840-4b4f-80be-d782cd7768a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:46.045429 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2b00cb5e-b840-4b4f-80be-d782cd7768a1 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.160260 Oct 20 09:05:46.045998 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 367/743] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:05:45 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments => generated 197 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:05:46.073422 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-46442b6c-3b6c-4318-ab14-9ef7d37423b7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.212539 Oct 20 09:05:46.073998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 377/744] 10.222.0.36 () {58 vars in 1125 bytes} [Thu Oct 20 09:05:45 2022] GET /compute/v2.1/servers => generated 343 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:46.085148 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-54d50819-123b-47db-aa4a-d12aa6f5be26 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:46.085755 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-54d50819-123b-47db-aa4a-d12aa6f5be26 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:46.092526 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-54d50819-123b-47db-aa4a-d12aa6f5be26 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:46.093010 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-54d50819-123b-47db-aa4a-d12aa6f5be26 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:46.093450 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-54d50819-123b-47db-aa4a-d12aa6f5be26 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:46.094140 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-54d50819-123b-47db-aa4a-d12aa6f5be26 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:46.094633 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-54d50819-123b-47db-aa4a-d12aa6f5be26 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:46.095064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-54d50819-123b-47db-aa4a-d12aa6f5be26 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:46.223581 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-54d50819-123b-47db-aa4a-d12aa6f5be26 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:46.231703 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-54d50819-123b-47db-aa4a-d12aa6f5be26 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1549 microversion: 2.1 time: 0.150257 Oct 20 09:05:46.232270 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 368/745] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:46 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1549 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:46.268933 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:46.269539 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:46.276364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:46.276937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:46.277376 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:46.278086 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:46.278561 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:46.278985 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:46.309495 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:46.309968 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:46.310421 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:46.310883 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lazy-loading 'fault' on Instance uuid ea0966d3-1311-4690-a69f-d3df935a76e1 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:05:46.351549 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:46.359304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f97fb44c-b571-4bed-96a6-e91ab6d4fe4c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 200 len: 1335 microversion: 2.1 time: 0.094121 Oct 20 09:05:46.368269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 378/746] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:46 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 1335 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:47.058293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-18c79a67-e0ce-43b8-94bc-95885b8d3a6e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:47.061060 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-18c79a67-e0ce-43b8-94bc-95885b8d3a6e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:47.067919 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18c79a67-e0ce-43b8-94bc-95885b8d3a6e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:47.068399 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18c79a67-e0ce-43b8-94bc-95885b8d3a6e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:47.068834 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18c79a67-e0ce-43b8-94bc-95885b8d3a6e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:47.069524 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18c79a67-e0ce-43b8-94bc-95885b8d3a6e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:47.069969 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18c79a67-e0ce-43b8-94bc-95885b8d3a6e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:47.070384 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-18c79a67-e0ce-43b8-94bc-95885b8d3a6e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:47.102029 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-18c79a67-e0ce-43b8-94bc-95885b8d3a6e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048285 Oct 20 09:05:47.102592 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 369/747] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:05:47 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:47.383339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:47.384119 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: ea0966d3-1311-4690-a69f-d3df935a76e1] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:47.401453 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:47.402062 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:47.402523 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:47.403220 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:47.403666 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:47.404081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:47.420080 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] HTTP exception thrown: Instance ea0966d3-1311-4690-a69f-d3df935a76e1 could not be found. Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Request method failure captured: Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 HTTP/1.1 Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ44IzvbrfISoSWQdaz7A4Y4mCB866TdY__L0_HKNgMuGtSn76PJ_HBHhaSlJVS78rXHSeDqK-8OZHSlR6XDrjto3KeVysuo0baUN07Q2QroH6vEp0-TETCDD7MgP6eUx6PZ5XfPA9gmR0Yq3sbUMXAjpaWW4IJsJYamevW56vxD4ZM Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: d24a1fc0e9344af181d49cfc58f0c88b Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:05:47.421722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/d24a1fc0e9344af181d49cfc58f0c88b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_d24a1fc0e9344af181d49cfc58f0c88b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/d24a1fc0e9344af181d49cfc58f0c88b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/d24a1fc0e9344af181d49cfc58f0c88b"}]}] Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: d24a1fc0e9344af181d49cfc58f0c88b Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-ServerRescueNegativeTestJSON-1468667757-project Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 7f932695a5994c5790d14c210ea65326 Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1468667757-project Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance ea0966d3-1311-4690-a69f-d3df935a76e1 could not be found. Oct 20 09:05:47.423418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': 'ea0966d3-1311-4690-a69f-d3df935a76e1'} Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ea0966d3-1311-4690-a69f-d3df935a76e1 could not be found. Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:05:47.424173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ea0966d3-1311-4690-a69f-d3df935a76e1 could not be found. Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ea0966d3-1311-4690-a69f-d3df935a76e1 could not be found. Oct 20 09:05:47.425327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:47.431099 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Returning 404 to user: Instance ea0966d3-1311-4690-a69f-d3df935a76e1 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:05:47.432172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5a0a58be-7faa-4d23-894e-7d9dede5e74d tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1" status: 404 len: 111 microversion: 2.1 time: 0.053507 Oct 20 09:05:47.432890 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 379/748] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:47 2022] GET /compute/v2.1/servers/ea0966d3-1311-4690-a69f-d3df935a76e1 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:05:47.441840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cf86b5a3-3d31-486d-85d4-71333f362920 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '.version_select at 0x7f5afba4f310>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:47.459946 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-cf86b5a3-3d31-486d-85d4-71333f362920 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1923229509" status: 202 len: 0 microversion: 2.1 time: 0.021960 Oct 20 09:05:47.460584 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 370/749] 10.222.0.36 () {58 vars in 1221 bytes} [Thu Oct 20 09:05:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1923229509 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:05:47.829247 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8affdf7b-c920-4f41-a8c8-cb7959bcb6f0 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e6fc3341-7c17-4457-942c-6564d78976e5"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:47.831657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8affdf7b-c920-4f41-a8c8-cb7959bcb6f0 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:47.837445 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8affdf7b-c920-4f41-a8c8-cb7959bcb6f0 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:47.837934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8affdf7b-c920-4f41-a8c8-cb7959bcb6f0 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:47.839023 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8affdf7b-c920-4f41-a8c8-cb7959bcb6f0 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:47.839713 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8affdf7b-c920-4f41-a8c8-cb7959bcb6f0 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:47.840419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8affdf7b-c920-4f41-a8c8-cb7959bcb6f0 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:47.840843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8affdf7b-c920-4f41-a8c8-cb7959bcb6f0 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:47.844524 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:48.116501 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9d94de4e-89af-4b69-a73a-da95f6dab2fd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:48.118981 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-9d94de4e-89af-4b69-a73a-da95f6dab2fd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:48.126852 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d94de4e-89af-4b69-a73a-da95f6dab2fd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:48.127265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d94de4e-89af-4b69-a73a-da95f6dab2fd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:48.127991 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d94de4e-89af-4b69-a73a-da95f6dab2fd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:48.128882 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d94de4e-89af-4b69-a73a-da95f6dab2fd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:48.129311 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d94de4e-89af-4b69-a73a-da95f6dab2fd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:48.129810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d94de4e-89af-4b69-a73a-da95f6dab2fd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:48.170035 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-9d94de4e-89af-4b69-a73a-da95f6dab2fd tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.059132 Oct 20 09:05:48.170798 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 371/750] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:05:48 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289/os-volume_attachments => generated 25 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 20 09:05:48.248140 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3b3a79bf-7d1d-4172-909b-719d07ce123e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:48.248616 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3b3a79bf-7d1d-4172-909b-719d07ce123e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:48.251334 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8affdf7b-c920-4f41-a8c8-cb7959bcb6f0 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "POST /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.425848 Oct 20 09:05:48.252041 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 380/751] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:05:47 2022] POST /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments => generated 194 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:48.255987 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3b3a79bf-7d1d-4172-909b-719d07ce123e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:48.256465 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3b3a79bf-7d1d-4172-909b-719d07ce123e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:48.256899 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3b3a79bf-7d1d-4172-909b-719d07ce123e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:48.257721 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3b3a79bf-7d1d-4172-909b-719d07ce123e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:48.258169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3b3a79bf-7d1d-4172-909b-719d07ce123e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:48.258616 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3b3a79bf-7d1d-4172-909b-719d07ce123e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:48.290144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3b3a79bf-7d1d-4172-909b-719d07ce123e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:05:48.372172 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3b3a79bf-7d1d-4172-909b-719d07ce123e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 204 len: 0 microversion: 2.1 time: 0.129555 Oct 20 09:05:48.372834 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 372/752] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:05:48 2022] DELETE /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 20 09:05:48.383606 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f136d293-970b-495d-881a-d85dddaa3687 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:48.384125 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f136d293-970b-495d-881a-d85dddaa3687 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:48.390668 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f136d293-970b-495d-881a-d85dddaa3687 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:48.391378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f136d293-970b-495d-881a-d85dddaa3687 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:48.391937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f136d293-970b-495d-881a-d85dddaa3687 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:48.392655 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f136d293-970b-495d-881a-d85dddaa3687 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:48.393091 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f136d293-970b-495d-881a-d85dddaa3687 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:48.393510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f136d293-970b-495d-881a-d85dddaa3687 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:48.532183 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f136d293-970b-495d-881a-d85dddaa3687 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:48.541931 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f136d293-970b-495d-881a-d85dddaa3687 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1682 microversion: 2.1 time: 0.163678 Oct 20 09:05:48.542532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 381/753] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:48 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1682 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:49.566527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2bbbb80c-bc1a-4bf5-bd0f-130895e5a58a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:49.567195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2bbbb80c-bc1a-4bf5-bd0f-130895e5a58a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:49.573947 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2bbbb80c-bc1a-4bf5-bd0f-130895e5a58a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:49.574424 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2bbbb80c-bc1a-4bf5-bd0f-130895e5a58a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:49.574881 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2bbbb80c-bc1a-4bf5-bd0f-130895e5a58a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:49.575557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2bbbb80c-bc1a-4bf5-bd0f-130895e5a58a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:49.576002 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2bbbb80c-bc1a-4bf5-bd0f-130895e5a58a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:49.576420 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2bbbb80c-bc1a-4bf5-bd0f-130895e5a58a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:49.713177 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2bbbb80c-bc1a-4bf5-bd0f-130895e5a58a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:49.721262 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2bbbb80c-bc1a-4bf5-bd0f-130895e5a58a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1682 microversion: 2.1 time: 0.159817 Oct 20 09:05:49.721838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 373/754] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:49 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1682 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:50.374994 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-97382dcb-61be-44d9-b2a6-5a555bd51665 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:50.375524 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-97382dcb-61be-44d9-b2a6-5a555bd51665 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:50.382341 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97382dcb-61be-44d9-b2a6-5a555bd51665 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:50.383228 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97382dcb-61be-44d9-b2a6-5a555bd51665 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:50.383336 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97382dcb-61be-44d9-b2a6-5a555bd51665 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:50.383988 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97382dcb-61be-44d9-b2a6-5a555bd51665 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:50.384436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97382dcb-61be-44d9-b2a6-5a555bd51665 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:50.384859 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97382dcb-61be-44d9-b2a6-5a555bd51665 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:50.414555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-97382dcb-61be-44d9-b2a6-5a555bd51665 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:05:50.500976 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-97382dcb-61be-44d9-b2a6-5a555bd51665 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "DELETE /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 204 len: 0 microversion: 2.1 time: 0.129326 Oct 20 09:05:50.502113 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 382/755] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:05:50 2022] DELETE /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:05:50.510868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-270ea8e1-559b-412f-a8b5-cfe48cee460b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:50.511749 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-270ea8e1-559b-412f-a8b5-cfe48cee460b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:50.518629 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-270ea8e1-559b-412f-a8b5-cfe48cee460b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:50.518869 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-270ea8e1-559b-412f-a8b5-cfe48cee460b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:50.519827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-270ea8e1-559b-412f-a8b5-cfe48cee460b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:50.520407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-270ea8e1-559b-412f-a8b5-cfe48cee460b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:50.521076 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-270ea8e1-559b-412f-a8b5-cfe48cee460b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:50.521512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-270ea8e1-559b-412f-a8b5-cfe48cee460b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:50.727052 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-270ea8e1-559b-412f-a8b5-cfe48cee460b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:50.735196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-270ea8e1-559b-412f-a8b5-cfe48cee460b tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1506 microversion: 2.1 time: 0.228527 Oct 20 09:05:50.735747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 374/756] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:50 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1506 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:50.744662 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7d482635-369d-4926-9d21-5831cf0df552 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:50.745914 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7d482635-369d-4926-9d21-5831cf0df552 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:50.752677 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d482635-369d-4926-9d21-5831cf0df552 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:50.753146 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d482635-369d-4926-9d21-5831cf0df552 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:50.753569 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d482635-369d-4926-9d21-5831cf0df552 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:50.754259 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d482635-369d-4926-9d21-5831cf0df552 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:50.754776 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d482635-369d-4926-9d21-5831cf0df552 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:50.755198 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d482635-369d-4926-9d21-5831cf0df552 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:50.883514 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7d482635-369d-4926-9d21-5831cf0df552 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:50.891306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7d482635-369d-4926-9d21-5831cf0df552 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1682 microversion: 2.1 time: 0.150832 Oct 20 09:05:50.891870 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 383/757] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:50 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1682 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:51.257037 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-7f6ef67d-0313-49f1-91c2-3f1fb2946e4d req-d6ae75ae-d4d8-45ef-991b-091587e14341 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "06d8b443-0b7d-4287-8469-eb4a35863289", "name": "network-vif-deleted", "tag": "db1c0050-0582-4d57-ba24-66fe0c293b1e"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:51.266422 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-7f6ef67d-0313-49f1-91c2-3f1fb2946e4d req-d6ae75ae-d4d8-45ef-991b-091587e14341 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:51.266864 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-7f6ef67d-0313-49f1-91c2-3f1fb2946e4d req-d6ae75ae-d4d8-45ef-991b-091587e14341 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:51.267297 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-7f6ef67d-0313-49f1-91c2-3f1fb2946e4d req-d6ae75ae-d4d8-45ef-991b-091587e14341 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:51.278993 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-7f6ef67d-0313-49f1-91c2-3f1fb2946e4d req-d6ae75ae-d4d8-45ef-991b-091587e14341 service nova] Creating event network-vif-deleted:db1c0050-0582-4d57-ba24-66fe0c293b1e for instance 06d8b443-0b7d-4287-8469-eb4a35863289 on hv-cinder-maste Oct 20 09:05:51.283851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-7f6ef67d-0313-49f1-91c2-3f1fb2946e4d req-d6ae75ae-d4d8-45ef-991b-091587e14341 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030449 Oct 20 09:05:51.284444 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 375/758] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:05:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:51.758554 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c820a69b-6ed7-440e-9655-6d55a82d93ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:51.759072 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c820a69b-6ed7-440e-9655-6d55a82d93ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:51.765929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c820a69b-6ed7-440e-9655-6d55a82d93ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:51.766399 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c820a69b-6ed7-440e-9655-6d55a82d93ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:51.766906 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c820a69b-6ed7-440e-9655-6d55a82d93ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:51.767589 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c820a69b-6ed7-440e-9655-6d55a82d93ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:51.768026 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c820a69b-6ed7-440e-9655-6d55a82d93ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:51.768438 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c820a69b-6ed7-440e-9655-6d55a82d93ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:51.899972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c820a69b-6ed7-440e-9655-6d55a82d93ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:51.907935 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c820a69b-6ed7-440e-9655-6d55a82d93ef tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1506 microversion: 2.1 time: 0.153279 Oct 20 09:05:51.908505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 384/759] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:51 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1506 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:51.915418 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b031e07e-6aba-470c-9877-7b0d9dd798e2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:51.916119 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b031e07e-6aba-470c-9877-7b0d9dd798e2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:51.922743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b031e07e-6aba-470c-9877-7b0d9dd798e2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:51.923233 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b031e07e-6aba-470c-9877-7b0d9dd798e2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:51.923670 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b031e07e-6aba-470c-9877-7b0d9dd798e2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:51.924390 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b031e07e-6aba-470c-9877-7b0d9dd798e2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:51.924833 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b031e07e-6aba-470c-9877-7b0d9dd798e2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:51.925276 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b031e07e-6aba-470c-9877-7b0d9dd798e2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:51.988962 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b031e07e-6aba-470c-9877-7b0d9dd798e2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:51.997598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b031e07e-6aba-470c-9877-7b0d9dd798e2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1614 microversion: 2.1 time: 0.086772 Oct 20 09:05:51.998237 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 376/760] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:51 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1614 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:52.930838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d139df05-9d3d-4058-8aa6-db7ea773d296 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:52.931430 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d139df05-9d3d-4058-8aa6-db7ea773d296 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:52.938390 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d139df05-9d3d-4058-8aa6-db7ea773d296 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:52.938904 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d139df05-9d3d-4058-8aa6-db7ea773d296 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:52.939344 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d139df05-9d3d-4058-8aa6-db7ea773d296 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:52.940022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d139df05-9d3d-4058-8aa6-db7ea773d296 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:52.940470 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d139df05-9d3d-4058-8aa6-db7ea773d296 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:52.940892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d139df05-9d3d-4058-8aa6-db7ea773d296 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:53.021201 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-43408ecf-8303-4ae9-bf15-e66b763d3a1d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:53.021835 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-43408ecf-8303-4ae9-bf15-e66b763d3a1d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:53.028536 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43408ecf-8303-4ae9-bf15-e66b763d3a1d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:53.029022 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43408ecf-8303-4ae9-bf15-e66b763d3a1d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:53.029668 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43408ecf-8303-4ae9-bf15-e66b763d3a1d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:53.030366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43408ecf-8303-4ae9-bf15-e66b763d3a1d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:53.030838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43408ecf-8303-4ae9-bf15-e66b763d3a1d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:53.031257 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43408ecf-8303-4ae9-bf15-e66b763d3a1d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:53.069344 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d139df05-9d3d-4058-8aa6-db7ea773d296 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:53.078225 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d139df05-9d3d-4058-8aa6-db7ea773d296 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1506 microversion: 2.1 time: 0.151095 Oct 20 09:05:53.079217 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 385/761] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:52 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1506 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:53.096758 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-43408ecf-8303-4ae9-bf15-e66b763d3a1d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:53.105277 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-43408ecf-8303-4ae9-bf15-e66b763d3a1d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 200 len: 1327 microversion: 2.1 time: 0.087421 Oct 20 09:05:53.105840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 377/762] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:53 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 1327 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:53.682075 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-cc4aeedb-17a5-4ead-87d8-9ef2d0690a81 req-d95824df-9ddf-4a8d-bc13-c221c8eecefb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0dd1dfdc-5e1e-4a08-a814-4f783857b2f7", "name": "network-vif-deleted", "tag": "0d706cc3-dd7f-4052-9a1e-5635f7716b26"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:53.692248 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-cc4aeedb-17a5-4ead-87d8-9ef2d0690a81 req-d95824df-9ddf-4a8d-bc13-c221c8eecefb service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:53.692806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-cc4aeedb-17a5-4ead-87d8-9ef2d0690a81 req-d95824df-9ddf-4a8d-bc13-c221c8eecefb service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:53.693308 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-cc4aeedb-17a5-4ead-87d8-9ef2d0690a81 req-d95824df-9ddf-4a8d-bc13-c221c8eecefb service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:53.704177 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-cc4aeedb-17a5-4ead-87d8-9ef2d0690a81 req-d95824df-9ddf-4a8d-bc13-c221c8eecefb service nova] Creating event network-vif-deleted:0d706cc3-dd7f-4052-9a1e-5635f7716b26 for instance 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 on hv-cinder-maste Oct 20 09:05:53.709044 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-cc4aeedb-17a5-4ead-87d8-9ef2d0690a81 req-d95824df-9ddf-4a8d-bc13-c221c8eecefb service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030650 Oct 20 09:05:53.709744 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 386/763] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:05:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:54.100995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:54.101606 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:54.109097 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:54.109586 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:54.110019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:54.110746 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:54.111371 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:54.111959 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:54.128888 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:54.129842 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:54.130397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:54.144929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:54.145416 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:54.146651 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:54.147315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:54.147761 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:54.147899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:54.148362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:54.148482 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:54.148712 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:54.149155 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lazy-loading 'fault' on Instance uuid 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:05:54.163862 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 HTTP/1.1 Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:05:54.165498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:05:54.166880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '06d8b443-0b7d-4287-8469-eb4a35863289'} Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:05:54.168166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:05:54.169319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:05:54.170355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:05:54.170355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-16e16417-560f-4961-a12a-ff8738c9fb15 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 404 len: 111 microversion: 2.1 time: 0.042844 Oct 20 09:05:54.170355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 387/764] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:54 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 20 09:05:54.177966 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6d6152dc-782b-438f-b947-064d875308f3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '.version_select at 0x7f5afb948b80>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:54.196315 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:54.198154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6d6152dc-782b-438f-b947-064d875308f3 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2061928950" status: 202 len: 0 microversion: 2.1 time: 0.025223 Oct 20 09:05:54.199043 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 388/765] 10.222.0.36 () {58 vars in 1221 bytes} [Thu Oct 20 09:05:54 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2061928950 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:05:54.208283 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-da2650f3-84a4-41b9-8398-a4662c9203b6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 200 len: 1311 microversion: 2.1 time: 0.111043 Oct 20 09:05:54.208854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 378/766] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:54 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 1311 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:54.806806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a6423c4e-6fbc-4d29-90a1-dde4537cc0fe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: .version_select at 0x7f5afb8064c0>, body: {"keypair": {"name": "tempest-keypair-437173038", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAvVOsNpafPE4T4/wAPpQTq2s67OLd/3zYsXXMX1a0JCwLaEbMe9C0X41oUjUsSaHdwyvgWGLWzny/agqk22utF7geNkA1TKFGSwtS16BdGA++JBnjvtJoyvbJhU3WQNoQ=="}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:54.823024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-a6423c4e-6fbc-4d29-90a1-dde4537cc0fe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:05:54.827517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-a6423c4e-6fbc-4d29-90a1-dde4537cc0fe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for user 8002ab26f6d64822aaa65aa730781b76 and project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'key_pairs'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:05:54.848462 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a6423c4e-6fbc-4d29-90a1-dde4537cc0fe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.045717 Oct 20 09:05:54.849044 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 389/767] 10.222.0.36 () {60 vars in 1159 bytes} [Thu Oct 20 09:05:54 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:05:55.232298 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:55.232856 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:55.239259 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:55.239745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:55.240185 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:55.240881 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:55.241381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:55.241767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:55.255875 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] HTTP exception thrown: Instance 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 could not be found. Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Request method failure captured: Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 HTTP/1.1 Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ44IzvbrfISoSWQdaz7A4Y4mCB866TdY__L0_HKNgMuGtSn76PJ_HBHhaSlJVS78rXHSeDqK-8OZHSlR6XDrjto3KeVysuo0baUN07Q2QroH6vEp0-TETCDD7MgP6eUx6PZ5XfPA9gmR0Yq3sbUMXAjpaWW4IJsJYamevW56vxD4ZM Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: d24a1fc0e9344af181d49cfc58f0c88b Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:05:55.257540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/d24a1fc0e9344af181d49cfc58f0c88b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_d24a1fc0e9344af181d49cfc58f0c88b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/d24a1fc0e9344af181d49cfc58f0c88b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/d24a1fc0e9344af181d49cfc58f0c88b"}]}] Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: d24a1fc0e9344af181d49cfc58f0c88b Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-ServerRescueNegativeTestJSON-1468667757-project Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 7f932695a5994c5790d14c210ea65326 Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1468667757-project Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 could not be found. Oct 20 09:05:55.259123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '0dd1dfdc-5e1e-4a08-a814-4f783857b2f7'} Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 could not be found. Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:05:55.260004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 could not be found. Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 could not be found. Oct 20 09:05:55.261407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:05:55.262667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Returning 404 to user: Instance 0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:05:55.262667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ef32becb-a103-4fd9-860e-b5aa10b796bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7" status: 404 len: 111 microversion: 2.1 time: 0.031059 Oct 20 09:05:55.262827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 379/768] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:55 2022] GET /compute/v2.1/servers/0dd1dfdc-5e1e-4a08-a814-4f783857b2f7 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:05:55.268887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8031ff76-4cc5-4f8d-bc56-ccf967ea19e1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:55.269443 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8031ff76-4cc5-4f8d-bc56-ccf967ea19e1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:55.275167 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8031ff76-4cc5-4f8d-bc56-ccf967ea19e1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:55.275652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8031ff76-4cc5-4f8d-bc56-ccf967ea19e1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:55.276085 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8031ff76-4cc5-4f8d-bc56-ccf967ea19e1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:55.276805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8031ff76-4cc5-4f8d-bc56-ccf967ea19e1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:55.277284 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8031ff76-4cc5-4f8d-bc56-ccf967ea19e1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:55.277701 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8031ff76-4cc5-4f8d-bc56-ccf967ea19e1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:55.304775 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8031ff76-4cc5-4f8d-bc56-ccf967ea19e1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:05:55.393064 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8031ff76-4cc5-4f8d-bc56-ccf967ea19e1 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "DELETE /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 204 len: 0 microversion: 2.1 time: 0.128618 Oct 20 09:05:55.394151 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 390/769] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:05:55 2022] DELETE /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:05:55.403125 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-99e2584a-a8f5-4780-a92d-15100e63ba17 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:55.403740 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-99e2584a-a8f5-4780-a92d-15100e63ba17 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:55.409692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99e2584a-a8f5-4780-a92d-15100e63ba17 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:55.410182 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99e2584a-a8f5-4780-a92d-15100e63ba17 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:55.410691 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99e2584a-a8f5-4780-a92d-15100e63ba17 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:55.411384 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99e2584a-a8f5-4780-a92d-15100e63ba17 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:55.411837 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99e2584a-a8f5-4780-a92d-15100e63ba17 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:55.412262 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-99e2584a-a8f5-4780-a92d-15100e63ba17 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:55.634209 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-99e2584a-a8f5-4780-a92d-15100e63ba17 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:55.641906 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-99e2584a-a8f5-4780-a92d-15100e63ba17 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.243263 Oct 20 09:05:55.642465 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 380/770] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:55 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:56.124378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-25464646", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b8327be3-f028-4ebc-9b25-81152867d29e"}], "security_groups": [{"name": "tempest-securitygroup--832184370"}], "key_name": "tempest-keypair-437173038", "user_data": "}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:56.132790 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:05:56.230885 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] validate_networks() for [('b8327be3-f028-4ebc-9b25-81152867d29e', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:05:56.631209 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:05:56.631800 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:05:56.632097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:05:56.632495 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:05:56.632893 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:05:56.642001 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:56.642486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:56.642971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:56.643682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:56.644123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:56.644533 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:56.661433 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:05:56.665240 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-abfbacd8-0ff7-4ff5-bb61-33b5aec66f71 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:56.665718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Getting quotas for user 8002ab26f6d64822aaa65aa730781b76 and project fcf51428b6bd4c428e1e01aadc6abe48. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:05:56.665831 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-abfbacd8-0ff7-4ff5-bb61-33b5aec66f71 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:56.672228 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abfbacd8-0ff7-4ff5-bb61-33b5aec66f71 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:56.672706 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abfbacd8-0ff7-4ff5-bb61-33b5aec66f71 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:56.673166 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abfbacd8-0ff7-4ff5-bb61-33b5aec66f71 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:56.673855 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abfbacd8-0ff7-4ff5-bb61-33b5aec66f71 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:56.674303 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abfbacd8-0ff7-4ff5-bb61-33b5aec66f71 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:56.674751 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-abfbacd8-0ff7-4ff5-bb61-33b5aec66f71 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:56.675947 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:05:56.678315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:05:56.725992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bf3a550e-697a-4897-b1b3-cda24dbc2d90 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.606217 Oct 20 09:05:56.726951 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 391/771] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:05:56 2022] POST /compute/v2.1/servers => generated 401 bytes in 608 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:05:56.739217 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1840dacf-cd95-4521-976d-659354fdbcaf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:56.740206 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1840dacf-cd95-4521-976d-659354fdbcaf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:56.800316 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1840dacf-cd95-4521-976d-659354fdbcaf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:56.810809 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1840dacf-cd95-4521-976d-659354fdbcaf tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1239 microversion: 2.1 time: 0.072444 Oct 20 09:05:56.811425 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 392/772] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:56 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1239 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:05:56.851580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-abfbacd8-0ff7-4ff5-bb61-33b5aec66f71 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:56.859464 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-abfbacd8-0ff7-4ff5-bb61-33b5aec66f71 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.198071 Oct 20 09:05:56.860032 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 381/773] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:56 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:05:57.836710 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-780ebe7a-0570-47f0-9160-168a98f38cd8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:57.837314 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-780ebe7a-0570-47f0-9160-168a98f38cd8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:57.844472 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-780ebe7a-0570-47f0-9160-168a98f38cd8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:57.844949 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-780ebe7a-0570-47f0-9160-168a98f38cd8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:57.845378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-780ebe7a-0570-47f0-9160-168a98f38cd8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:57.846062 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-780ebe7a-0570-47f0-9160-168a98f38cd8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:57.846533 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-780ebe7a-0570-47f0-9160-168a98f38cd8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:57.846951 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-780ebe7a-0570-47f0-9160-168a98f38cd8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:57.882384 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-efc5b490-b6cf-4296-9b9f-a755ca62dfc0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:57.883016 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-efc5b490-b6cf-4296-9b9f-a755ca62dfc0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:57.889500 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efc5b490-b6cf-4296-9b9f-a755ca62dfc0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:57.889982 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efc5b490-b6cf-4296-9b9f-a755ca62dfc0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:57.890412 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efc5b490-b6cf-4296-9b9f-a755ca62dfc0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:57.891123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efc5b490-b6cf-4296-9b9f-a755ca62dfc0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:57.891569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efc5b490-b6cf-4296-9b9f-a755ca62dfc0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:57.891987 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efc5b490-b6cf-4296-9b9f-a755ca62dfc0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:57.907591 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-780ebe7a-0570-47f0-9160-168a98f38cd8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:57.915278 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-780ebe7a-0570-47f0-9160-168a98f38cd8 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1243 microversion: 2.1 time: 0.082376 Oct 20 09:05:57.915783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 393/774] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:57 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1243 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:58.023410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-efc5b490-b6cf-4296-9b9f-a755ca62dfc0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:58.031552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-efc5b490-b6cf-4296-9b9f-a755ca62dfc0 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.152943 Oct 20 09:05:58.032149 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 382/775] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:57 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:05:58.937200 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9855139a-66b7-4089-b828-19e8ed4bce4d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:58.937802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9855139a-66b7-4089-b828-19e8ed4bce4d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:58.944335 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9855139a-66b7-4089-b828-19e8ed4bce4d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:58.945031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9855139a-66b7-4089-b828-19e8ed4bce4d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:58.945456 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9855139a-66b7-4089-b828-19e8ed4bce4d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:58.946140 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9855139a-66b7-4089-b828-19e8ed4bce4d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:58.946620 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9855139a-66b7-4089-b828-19e8ed4bce4d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:58.947030 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9855139a-66b7-4089-b828-19e8ed4bce4d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:59.013645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9855139a-66b7-4089-b828-19e8ed4bce4d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:59.021253 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9855139a-66b7-4089-b828-19e8ed4bce4d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1243 microversion: 2.1 time: 0.087943 Oct 20 09:05:59.021817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 394/776] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:58 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1243 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:59.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-608caf45-5393-43a8-9932-b6fc2f93a20c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:05:59.055218 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-608caf45-5393-43a8-9932-b6fc2f93a20c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:05:59.061520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-608caf45-5393-43a8-9932-b6fc2f93a20c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:59.061994 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-608caf45-5393-43a8-9932-b6fc2f93a20c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:59.062422 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-608caf45-5393-43a8-9932-b6fc2f93a20c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:59.063122 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-608caf45-5393-43a8-9932-b6fc2f93a20c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:59.063567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-608caf45-5393-43a8-9932-b6fc2f93a20c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:59.063988 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-608caf45-5393-43a8-9932-b6fc2f93a20c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:59.193230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-608caf45-5393-43a8-9932-b6fc2f93a20c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:05:59.201606 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-608caf45-5393-43a8-9932-b6fc2f93a20c tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1520 microversion: 2.1 time: 0.150810 Oct 20 09:05:59.202170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 383/777] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:05:59 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1520 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:05:59.835604 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-56c05732-b039-4f98-beaf-39c78dd9bd78 req-97e371ef-5c55-4114-99e5-1ca4224517fb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cabc1ab-03aa-4ff2-be06-d99f9efc6150", "name": "network-vif-deleted", "tag": "761aed46-611d-4ca6-9fc5-7334f3e86a8f"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:05:59.849343 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-56c05732-b039-4f98-beaf-39c78dd9bd78 req-97e371ef-5c55-4114-99e5-1ca4224517fb service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:05:59.850008 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-56c05732-b039-4f98-beaf-39c78dd9bd78 req-97e371ef-5c55-4114-99e5-1ca4224517fb service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:05:59.850919 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-56c05732-b039-4f98-beaf-39c78dd9bd78 req-97e371ef-5c55-4114-99e5-1ca4224517fb service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:05:59.864014 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-56c05732-b039-4f98-beaf-39c78dd9bd78 req-97e371ef-5c55-4114-99e5-1ca4224517fb service nova] Creating event network-vif-deleted:761aed46-611d-4ca6-9fc5-7334f3e86a8f for instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 on hv-cinder-maste Oct 20 09:05:59.868892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-56c05732-b039-4f98-beaf-39c78dd9bd78 req-97e371ef-5c55-4114-99e5-1ca4224517fb service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038389 Oct 20 09:05:59.869485 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 395/778] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:05:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:00.044446 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-230be89a-46f4-4762-bfb0-5a3f8f929860 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:00.045314 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-230be89a-46f4-4762-bfb0-5a3f8f929860 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:00.052293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-230be89a-46f4-4762-bfb0-5a3f8f929860 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:00.053154 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-230be89a-46f4-4762-bfb0-5a3f8f929860 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:00.053671 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-230be89a-46f4-4762-bfb0-5a3f8f929860 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:00.054465 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-230be89a-46f4-4762-bfb0-5a3f8f929860 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:00.055027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-230be89a-46f4-4762-bfb0-5a3f8f929860 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:00.056316 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-230be89a-46f4-4762-bfb0-5a3f8f929860 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:00.127561 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-230be89a-46f4-4762-bfb0-5a3f8f929860 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:00.133690 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-620f5157-cd12-4ac4-9ecb-b08ebbdb5c61 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:00.134601 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-620f5157-cd12-4ac4-9ecb-b08ebbdb5c61 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:00.136366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-230be89a-46f4-4762-bfb0-5a3f8f929860 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1291 microversion: 2.1 time: 0.096277 Oct 20 09:06:00.142906 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 384/779] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:00 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1291 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:00.143025 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-620f5157-cd12-4ac4-9ecb-b08ebbdb5c61 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:00.143025 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-620f5157-cd12-4ac4-9ecb-b08ebbdb5c61 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:00.143025 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-620f5157-cd12-4ac4-9ecb-b08ebbdb5c61 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:00.143025 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-620f5157-cd12-4ac4-9ecb-b08ebbdb5c61 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:00.143378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-620f5157-cd12-4ac4-9ecb-b08ebbdb5c61 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:00.143707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-620f5157-cd12-4ac4-9ecb-b08ebbdb5c61 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:00.224750 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:00.225355 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:00.232945 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:00.233438 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:00.233881 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:00.234618 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:00.235301 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:00.236053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:00.269078 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:00.269626 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:00.270286 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:00.270772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lazy-loading 'fault' on Instance uuid 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:06:00.288320 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-620f5157-cd12-4ac4-9ecb-b08ebbdb5c61 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:00.300785 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-620f5157-cd12-4ac4-9ecb-b08ebbdb5c61 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1595 microversion: 2.1 time: 0.171143 Oct 20 09:06:00.302801 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 396/780] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:00 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1595 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:00.315435 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f932695a5994c5790d14c210ea65326', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd24a1fc0e9344af181d49cfc58f0c88b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:00.323791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ffcbd049-264d-4fbe-a9b3-ec5f19f471bc tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 200 len: 1312 microversion: 2.1 time: 0.102432 Oct 20 09:06:00.324358 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 385/781] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:00 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 1312 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:01.159697 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5be76406-adad-42b6-9062-ef2b433e9925 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:01.160419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5be76406-adad-42b6-9062-ef2b433e9925 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:01.168336 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5be76406-adad-42b6-9062-ef2b433e9925 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:01.168929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5be76406-adad-42b6-9062-ef2b433e9925 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:01.169421 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5be76406-adad-42b6-9062-ef2b433e9925 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:01.170525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5be76406-adad-42b6-9062-ef2b433e9925 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:01.171147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5be76406-adad-42b6-9062-ef2b433e9925 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:01.171804 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5be76406-adad-42b6-9062-ef2b433e9925 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:01.243188 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5be76406-adad-42b6-9062-ef2b433e9925 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:01.246404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:01.253977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5be76406-adad-42b6-9062-ef2b433e9925 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1297 microversion: 2.1 time: 0.098143 Oct 20 09:06:01.254972 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 397/782] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:01 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1297 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:01.347068 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:01.347773 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] [instance: 6cabc1ab-03aa-4ff2-be06-d99f9efc6150] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:01.350203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=82,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:01.356222 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:01.356788 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:01.357296 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:01.358062 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:01.358614 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:01.359112 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:01.375685 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] HTTP exception thrown: Instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 could not be found. Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Request method failure captured: Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 HTTP/1.1 Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ44IzvbrfISoSWQdaz7A4Y4mCB866TdY__L0_HKNgMuGtSn76PJ_HBHhaSlJVS78rXHSeDqK-8OZHSlR6XDrjto3KeVysuo0baUN07Q2QroH6vEp0-TETCDD7MgP6eUx6PZ5XfPA9gmR0Yq3sbUMXAjpaWW4IJsJYamevW56vxD4ZM Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: d24a1fc0e9344af181d49cfc58f0c88b Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:06:01.377641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/d24a1fc0e9344af181d49cfc58f0c88b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_d24a1fc0e9344af181d49cfc58f0c88b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/d24a1fc0e9344af181d49cfc58f0c88b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/d24a1fc0e9344af181d49cfc58f0c88b"}]}] Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: d24a1fc0e9344af181d49cfc58f0c88b Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1468667757 Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-ServerRescueNegativeTestJSON-1468667757-project Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 7f932695a5994c5790d14c210ea65326 Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1468667757-project Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 could not be found. Oct 20 09:06:01.379128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '6cabc1ab-03aa-4ff2-be06-d99f9efc6150'} Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 could not be found. Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:06:01.379953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 could not be found. Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 could not be found. Oct 20 09:06:01.381186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:01.382250 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] Returning 404 to user: Instance 6cabc1ab-03aa-4ff2-be06-d99f9efc6150 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:06:01.382250 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4ebd2581-80b3-434f-9166-c37d762b66e6 tempest-ServerRescueNegativeTestJSON-1468667757 tempest-ServerRescueNegativeTestJSON-1468667757-project] 10.222.0.36 "GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150" status: 404 len: 111 microversion: 2.1 time: 0.035898 Oct 20 09:06:01.383650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 386/783] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:01 2022] GET /compute/v2.1/servers/6cabc1ab-03aa-4ff2-be06-d99f9efc6150 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:06:02.277747 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-cbfebb1a-4b36-4dde-8306-170abc5247db tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:02.282235 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-cbfebb1a-4b36-4dde-8306-170abc5247db tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:02.290658 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbfebb1a-4b36-4dde-8306-170abc5247db tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:02.291319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbfebb1a-4b36-4dde-8306-170abc5247db tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:02.292686 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbfebb1a-4b36-4dde-8306-170abc5247db tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:02.294423 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbfebb1a-4b36-4dde-8306-170abc5247db tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:02.295015 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbfebb1a-4b36-4dde-8306-170abc5247db tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:02.295540 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-cbfebb1a-4b36-4dde-8306-170abc5247db tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:02.723816 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-cbfebb1a-4b36-4dde-8306-170abc5247db tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:02.731974 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-cbfebb1a-4b36-4dde-8306-170abc5247db tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1364 microversion: 2.1 time: 0.458107 Oct 20 09:06:02.732541 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 398/784] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:02 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1364 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:03.135875 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-13810fb2-2c95-4a7d-8d67-29ac3440b0b6 req-2e73400f-eae0-4680-874c-2d47f48b3065 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03c3c04f-2850-480f-8d15-6f15327734e0", "tag": "3c5f5de7-af1f-4408-ba7d-6be5f0344198"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:03.154952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-13810fb2-2c95-4a7d-8d67-29ac3440b0b6 req-2e73400f-eae0-4680-874c-2d47f48b3065 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:03.155430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-13810fb2-2c95-4a7d-8d67-29ac3440b0b6 req-2e73400f-eae0-4680-874c-2d47f48b3065 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:03.158745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-13810fb2-2c95-4a7d-8d67-29ac3440b0b6 req-2e73400f-eae0-4680-874c-2d47f48b3065 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:03.191373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-13810fb2-2c95-4a7d-8d67-29ac3440b0b6 req-2e73400f-eae0-4680-874c-2d47f48b3065 service nova] Creating event network-changed:3c5f5de7-af1f-4408-ba7d-6be5f0344198 for instance 03c3c04f-2850-480f-8d15-6f15327734e0 on hv-cinder-maste Oct 20 09:06:03.204752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-13810fb2-2c95-4a7d-8d67-29ac3440b0b6 req-2e73400f-eae0-4680-874c-2d47f48b3065 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071744 Oct 20 09:06:03.206822 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 387/785] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:03.755341 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7f867f53-ca4b-4abb-98e1-1378d05abd11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:03.755953 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7f867f53-ca4b-4abb-98e1-1378d05abd11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:03.767989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f867f53-ca4b-4abb-98e1-1378d05abd11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:03.775366 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f867f53-ca4b-4abb-98e1-1378d05abd11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:03.775825 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f867f53-ca4b-4abb-98e1-1378d05abd11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:03.777186 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f867f53-ca4b-4abb-98e1-1378d05abd11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:03.777650 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f867f53-ca4b-4abb-98e1-1378d05abd11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:03.778082 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f867f53-ca4b-4abb-98e1-1378d05abd11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:04.247454 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7f867f53-ca4b-4abb-98e1-1378d05abd11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:04.257208 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7f867f53-ca4b-4abb-98e1-1378d05abd11 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1364 microversion: 2.1 time: 0.505714 Oct 20 09:06:04.280093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 399/786] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:03 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1364 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:05.258789 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c553b57e-b386-46f2-84eb-53be9629dcc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1515462211"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:05.259509 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c553b57e-b386-46f2-84eb-53be9629dcc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:05.266320 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c553b57e-b386-46f2-84eb-53be9629dcc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.266854 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c553b57e-b386-46f2-84eb-53be9629dcc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.267375 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c553b57e-b386-46f2-84eb-53be9629dcc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:05.268206 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c553b57e-b386-46f2-84eb-53be9629dcc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.268744 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c553b57e-b386-46f2-84eb-53be9629dcc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.269247 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c553b57e-b386-46f2-84eb-53be9629dcc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:05.287538 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-6c54007d-775e-4b57-9ce7-48b47b7bd360 req-f4e33dfb-198d-405e-93c9-421fa6130ce0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c5dba102-6a02-40f4-a502-d1fc1cdab301", "tag": "a91a700e-d01e-4c84-8f53-4255f46188de"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:05.298419 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6c54007d-775e-4b57-9ce7-48b47b7bd360 req-f4e33dfb-198d-405e-93c9-421fa6130ce0 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.298936 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6c54007d-775e-4b57-9ce7-48b47b7bd360 req-f4e33dfb-198d-405e-93c9-421fa6130ce0 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.299356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6c54007d-775e-4b57-9ce7-48b47b7bd360 req-f4e33dfb-198d-405e-93c9-421fa6130ce0 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:05.310133 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-6c54007d-775e-4b57-9ce7-48b47b7bd360 req-f4e33dfb-198d-405e-93c9-421fa6130ce0 service nova] Creating event network-changed:a91a700e-d01e-4c84-8f53-4255f46188de for instance c5dba102-6a02-40f4-a502-d1fc1cdab301 on hv-cinder-maste Oct 20 09:06:05.315331 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-6c54007d-775e-4b57-9ce7-48b47b7bd360 req-f4e33dfb-198d-405e-93c9-421fa6130ce0 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.093407 Oct 20 09:06:05.315924 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 388/787] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:05.321099 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-de507eee-8003-4958-8502-7d68a27b67ed tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:05.321786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-de507eee-8003-4958-8502-7d68a27b67ed tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:05.329458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de507eee-8003-4958-8502-7d68a27b67ed tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.329915 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de507eee-8003-4958-8502-7d68a27b67ed tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.330346 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de507eee-8003-4958-8502-7d68a27b67ed tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:05.331091 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de507eee-8003-4958-8502-7d68a27b67ed tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.331724 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de507eee-8003-4958-8502-7d68a27b67ed tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.332304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de507eee-8003-4958-8502-7d68a27b67ed tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:05.486007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-de507eee-8003-4958-8502-7d68a27b67ed tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:05.498737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-de507eee-8003-4958-8502-7d68a27b67ed tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1364 microversion: 2.1 time: 0.181354 Oct 20 09:06:05.499479 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 389/788] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:05 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1364 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:05.505925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.network.security_group_api [None req-c553b57e-b386-46f2-84eb-53be9629dcc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Adding security group 8dd08cd5-03a3-4623-a8b2-414b5852ea61 to port 3c5f5de7-af1f-4408-ba7d-6be5f0344198 Oct 20 09:06:05.545847 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ab82b1f0-c3aa-4400-b491-f8d6d956daf4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:05.548182 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ab82b1f0-c3aa-4400-b491-f8d6d956daf4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:05.554572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ab82b1f0-c3aa-4400-b491-f8d6d956daf4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.555073 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ab82b1f0-c3aa-4400-b491-f8d6d956daf4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.555507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ab82b1f0-c3aa-4400-b491-f8d6d956daf4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:05.556203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ab82b1f0-c3aa-4400-b491-f8d6d956daf4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.556659 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ab82b1f0-c3aa-4400-b491-f8d6d956daf4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.557087 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ab82b1f0-c3aa-4400-b491-f8d6d956daf4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:05.587866 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ab82b1f0-c3aa-4400-b491-f8d6d956daf4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046486 Oct 20 09:06:05.588436 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 390/789] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:06:05 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 20 09:06:05.655920 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-61a462c0-25b5-45fd-aac6-b76398784cfd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:05.656419 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-61a462c0-25b5-45fd-aac6-b76398784cfd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:05.664324 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-61a462c0-25b5-45fd-aac6-b76398784cfd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.674284 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-61a462c0-25b5-45fd-aac6-b76398784cfd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.675078 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-61a462c0-25b5-45fd-aac6-b76398784cfd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:05.676528 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-61a462c0-25b5-45fd-aac6-b76398784cfd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.677106 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-61a462c0-25b5-45fd-aac6-b76398784cfd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.677636 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-61a462c0-25b5-45fd-aac6-b76398784cfd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:05.716357 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-61a462c0-25b5-45fd-aac6-b76398784cfd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:06:05.842006 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-61a462c0-25b5-45fd-aac6-b76398784cfd tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "DELETE /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 204 len: 0 microversion: 2.1 time: 0.190185 Oct 20 09:06:05.849781 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 391/790] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:06:05 2022] DELETE /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 20 09:06:05.859446 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a3779571-b254-44fe-9d27-e35b348c3fc4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:05.860540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a3779571-b254-44fe-9d27-e35b348c3fc4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:05.868684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3779571-b254-44fe-9d27-e35b348c3fc4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.869519 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3779571-b254-44fe-9d27-e35b348c3fc4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.870899 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3779571-b254-44fe-9d27-e35b348c3fc4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:05.871896 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3779571-b254-44fe-9d27-e35b348c3fc4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:05.873258 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3779571-b254-44fe-9d27-e35b348c3fc4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:05.873890 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a3779571-b254-44fe-9d27-e35b348c3fc4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:06.179806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a3779571-b254-44fe-9d27-e35b348c3fc4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:06.192344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a3779571-b254-44fe-9d27-e35b348c3fc4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1673 microversion: 2.1 time: 0.340572 Oct 20 09:06:06.209466 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 392/791] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:05 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1673 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:06.284786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c553b57e-b386-46f2-84eb-53be9629dcc7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "POST /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/action" status: 202 len: 0 microversion: 2.1 time: 1.029804 Oct 20 09:06:06.286151 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 400/792] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:06:05 2022] POST /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/action => generated 0 bytes in 1032 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:06:06.294995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6c597b0a-3c04-4bab-a4af-baefa9ffa08a tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:06.296345 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6c597b0a-3c04-4bab-a4af-baefa9ffa08a tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:06.297574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-d4abadc3-a075-47f9-b2f8-0738ec014776 req-eefd80dd-7aa0-48cc-bdef-e88c2315a544 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03c3c04f-2850-480f-8d15-6f15327734e0", "tag": "3c5f5de7-af1f-4408-ba7d-6be5f0344198"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:06.307055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6c597b0a-3c04-4bab-a4af-baefa9ffa08a tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:06.309678 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6c597b0a-3c04-4bab-a4af-baefa9ffa08a tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:06.310331 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6c597b0a-3c04-4bab-a4af-baefa9ffa08a tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:06.311239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6c597b0a-3c04-4bab-a4af-baefa9ffa08a tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:06.311830 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-d4abadc3-a075-47f9-b2f8-0738ec014776 req-eefd80dd-7aa0-48cc-bdef-e88c2315a544 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:06.312379 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-d4abadc3-a075-47f9-b2f8-0738ec014776 req-eefd80dd-7aa0-48cc-bdef-e88c2315a544 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:06.312880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-d4abadc3-a075-47f9-b2f8-0738ec014776 req-eefd80dd-7aa0-48cc-bdef-e88c2315a544 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:06.314863 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6c597b0a-3c04-4bab-a4af-baefa9ffa08a tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:06.315308 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6c597b0a-3c04-4bab-a4af-baefa9ffa08a tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:06.334888 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-d4abadc3-a075-47f9-b2f8-0738ec014776 req-eefd80dd-7aa0-48cc-bdef-e88c2315a544 service nova] Creating event network-changed:3c5f5de7-af1f-4408-ba7d-6be5f0344198 for instance 03c3c04f-2850-480f-8d15-6f15327734e0 on hv-cinder-maste Oct 20 09:06:06.343288 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-d4abadc3-a075-47f9-b2f8-0738ec014776 req-eefd80dd-7aa0-48cc-bdef-e88c2315a544 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050034 Oct 20 09:06:06.344230 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 401/793] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:06.523317 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-aa6133a6-da8e-47b3-a51a-91ef660834c2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:06.525612 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-aa6133a6-da8e-47b3-a51a-91ef660834c2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:06.535884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa6133a6-da8e-47b3-a51a-91ef660834c2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:06.536791 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa6133a6-da8e-47b3-a51a-91ef660834c2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:06.537377 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa6133a6-da8e-47b3-a51a-91ef660834c2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:06.549296 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa6133a6-da8e-47b3-a51a-91ef660834c2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:06.549296 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa6133a6-da8e-47b3-a51a-91ef660834c2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:06.549296 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-aa6133a6-da8e-47b3-a51a-91ef660834c2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:06.627677 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6c597b0a-3c04-4bab-a4af-baefa9ffa08a tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:06.637159 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6c597b0a-3c04-4bab-a4af-baefa9ffa08a tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1761 microversion: 2.1 time: 0.346620 Oct 20 09:06:06.651969 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 393/794] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:06 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1761 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:06.657581 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:06.719138 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-aa6133a6-da8e-47b3-a51a-91ef660834c2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:06.731799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-aa6133a6-da8e-47b3-a51a-91ef660834c2 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1364 microversion: 2.1 time: 0.213320 Oct 20 09:06:06.732745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 402/795] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:06 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1364 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:07.223967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cc648b44-274a-42a3-bdf2-0bfedc0650cb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:07.224575 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-cc648b44-274a-42a3-bdf2-0bfedc0650cb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:07.231931 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc648b44-274a-42a3-bdf2-0bfedc0650cb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:07.232417 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc648b44-274a-42a3-bdf2-0bfedc0650cb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:07.233025 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc648b44-274a-42a3-bdf2-0bfedc0650cb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:07.233811 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc648b44-274a-42a3-bdf2-0bfedc0650cb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:07.234267 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc648b44-274a-42a3-bdf2-0bfedc0650cb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:07.234952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc648b44-274a-42a3-bdf2-0bfedc0650cb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:07.371435 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-cc648b44-274a-42a3-bdf2-0bfedc0650cb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:07.382623 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-cc648b44-274a-42a3-bdf2-0bfedc0650cb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1673 microversion: 2.1 time: 0.162974 Oct 20 09:06:07.383204 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 394/796] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:07 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1673 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:07.760963 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6caa7078-1270-4017-b597-6424c942af69 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:07.762109 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6caa7078-1270-4017-b597-6424c942af69 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:07.764555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=12 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:07.770554 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6caa7078-1270-4017-b597-6424c942af69 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:07.771041 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6caa7078-1270-4017-b597-6424c942af69 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:07.771477 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6caa7078-1270-4017-b597-6424c942af69 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:07.772168 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6caa7078-1270-4017-b597-6424c942af69 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:07.772975 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6caa7078-1270-4017-b597-6424c942af69 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:07.773689 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6caa7078-1270-4017-b597-6424c942af69 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:07.909537 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6caa7078-1270-4017-b597-6424c942af69 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:07.918061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6caa7078-1270-4017-b597-6424c942af69 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1364 microversion: 2.1 time: 0.161761 Oct 20 09:06:07.918849 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 403/797] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:07 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1364 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:08.406086 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-89311384-1572-4148-a3af-e912ad1b7d5c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:08.408883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-89311384-1572-4148-a3af-e912ad1b7d5c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:08.413561 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89311384-1572-4148-a3af-e912ad1b7d5c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:08.414291 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89311384-1572-4148-a3af-e912ad1b7d5c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:08.415015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89311384-1572-4148-a3af-e912ad1b7d5c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:08.415896 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89311384-1572-4148-a3af-e912ad1b7d5c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:08.417104 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89311384-1572-4148-a3af-e912ad1b7d5c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:08.417737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-89311384-1572-4148-a3af-e912ad1b7d5c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:08.440289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:08.723300 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-89311384-1572-4148-a3af-e912ad1b7d5c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:08.732216 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-89311384-1572-4148-a3af-e912ad1b7d5c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1673 microversion: 2.1 time: 0.329833 Oct 20 09:06:08.733349 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 395/798] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:08 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1673 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:08.941226 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b0af3c63-6d60-41bd-a9c5-e01271801cf9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:08.941840 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b0af3c63-6d60-41bd-a9c5-e01271801cf9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:08.948823 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0af3c63-6d60-41bd-a9c5-e01271801cf9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:08.949306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0af3c63-6d60-41bd-a9c5-e01271801cf9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:08.949741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0af3c63-6d60-41bd-a9c5-e01271801cf9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:08.950437 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0af3c63-6d60-41bd-a9c5-e01271801cf9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:08.950917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0af3c63-6d60-41bd-a9c5-e01271801cf9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:08.951342 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0af3c63-6d60-41bd-a9c5-e01271801cf9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:09.080969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b0af3c63-6d60-41bd-a9c5-e01271801cf9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:09.088863 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b0af3c63-6d60-41bd-a9c5-e01271801cf9 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1364 microversion: 2.1 time: 0.151540 Oct 20 09:06:09.089429 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 404/799] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:08 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1364 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:09.389437 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-24bdfbe8-b5c5-4776-9b5c-ec991dd4644c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:09.389933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-24bdfbe8-b5c5-4776-9b5c-ec991dd4644c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:09.396404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-24bdfbe8-b5c5-4776-9b5c-ec991dd4644c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:09.396998 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-24bdfbe8-b5c5-4776-9b5c-ec991dd4644c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:09.397539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-24bdfbe8-b5c5-4776-9b5c-ec991dd4644c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:09.398761 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-24bdfbe8-b5c5-4776-9b5c-ec991dd4644c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:09.399125 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-24bdfbe8-b5c5-4776-9b5c-ec991dd4644c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:09.399621 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-24bdfbe8-b5c5-4776-9b5c-ec991dd4644c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:09.424952 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-24bdfbe8-b5c5-4776-9b5c-ec991dd4644c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/ips" status: 200 len: 152 microversion: 2.20 time: 0.205987 Oct 20 09:06:09.432014 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 396/800] 10.222.0.36 () {60 vars in 1289 bytes} [Thu Oct 20 09:06:09 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/ips => generated 152 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 20 09:06:09.442747 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-265343ab-7314-49c8-8f75-131d45a4b7f0 req-f9a2464a-97a2-428c-9f83-16bae51d9fdd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "26ae0069-be67-4921-aca3-4cfae5214cf5", "name": "network-vif-deleted", "tag": "ead535c7-97fc-458e-b3a8-642e6ca66f0a"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:09.459234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:09.461550 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-265343ab-7314-49c8-8f75-131d45a4b7f0 req-f9a2464a-97a2-428c-9f83-16bae51d9fdd service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:09.462094 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-265343ab-7314-49c8-8f75-131d45a4b7f0 req-f9a2464a-97a2-428c-9f83-16bae51d9fdd service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:09.462551 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-265343ab-7314-49c8-8f75-131d45a4b7f0 req-f9a2464a-97a2-428c-9f83-16bae51d9fdd service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:09.484574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-265343ab-7314-49c8-8f75-131d45a4b7f0 req-f9a2464a-97a2-428c-9f83-16bae51d9fdd service nova] Creating event network-vif-deleted:ead535c7-97fc-458e-b3a8-642e6ca66f0a for instance 26ae0069-be67-4921-aca3-4cfae5214cf5 on hv-cinder-maste Oct 20 09:06:09.490419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-265343ab-7314-49c8-8f75-131d45a4b7f0 req-f9a2464a-97a2-428c-9f83-16bae51d9fdd service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054653 Oct 20 09:06:09.492246 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 405/801] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:09.759683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:09.761976 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:09.779823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:09.779823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:09.779823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:09.779823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:09.780221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:09.780221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:09.826275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:09.827012 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:09.827611 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:09.828229 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lazy-loading 'fault' on Instance uuid 26ae0069-be67-4921-aca3-4cfae5214cf5 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:06:09.892224 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:09.903092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-14a9061d-1219-45ae-8c19-672f617863a3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 200 len: 1326 microversion: 2.1 time: 0.149223 Oct 20 09:06:09.904462 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 397/802] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:09 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 1326 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:10.120548 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-4f8123a1-3f13-4781-be38-086ebbbd3d3c req-4f7fe81f-48a2-4e7e-acaa-9bf15c377e7c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c5dba102-6a02-40f4-a502-d1fc1cdab301", "name": "network-vif-plugged", "status": "completed", "tag": "a91a700e-d01e-4c84-8f53-4255f46188de"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:10.121643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a5a4ad3c-388e-4fb3-8348-4c8339ecc769 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:10.125383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a5a4ad3c-388e-4fb3-8348-4c8339ecc769 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:10.136030 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a4ad3c-388e-4fb3-8348-4c8339ecc769 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:10.136442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a4ad3c-388e-4fb3-8348-4c8339ecc769 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:10.136878 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a4ad3c-388e-4fb3-8348-4c8339ecc769 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:10.141503 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-4f8123a1-3f13-4781-be38-086ebbbd3d3c req-4f7fe81f-48a2-4e7e-acaa-9bf15c377e7c service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:10.141977 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-4f8123a1-3f13-4781-be38-086ebbbd3d3c req-4f7fe81f-48a2-4e7e-acaa-9bf15c377e7c service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:10.142404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-4f8123a1-3f13-4781-be38-086ebbbd3d3c req-4f7fe81f-48a2-4e7e-acaa-9bf15c377e7c service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:10.142576 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a4ad3c-388e-4fb3-8348-4c8339ecc769 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:10.143224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a4ad3c-388e-4fb3-8348-4c8339ecc769 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:10.143651 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5a4ad3c-388e-4fb3-8348-4c8339ecc769 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:10.177200 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-4f8123a1-3f13-4781-be38-086ebbbd3d3c req-4f7fe81f-48a2-4e7e-acaa-9bf15c377e7c service nova] Creating event network-vif-plugged:a91a700e-d01e-4c84-8f53-4255f46188de for instance c5dba102-6a02-40f4-a502-d1fc1cdab301 on hv-cinder-maste Oct 20 09:06:10.186640 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-4f8123a1-3f13-4781-be38-086ebbbd3d3c req-4f7fe81f-48a2-4e7e-acaa-9bf15c377e7c service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075659 Oct 20 09:06:10.188215 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 398/803] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:10.375427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a5a4ad3c-388e-4fb3-8348-4c8339ecc769 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:10.387814 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a5a4ad3c-388e-4fb3-8348-4c8339ecc769 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1364 microversion: 2.1 time: 0.278357 Oct 20 09:06:10.388406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 406/804] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:10 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1364 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:10.928714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:10.929414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:10.935533 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:10.936010 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:10.936442 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:10.937123 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:10.937570 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:10.937990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:10.951428 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] HTTP exception thrown: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Request method failure captured: Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 HTTP/1.1 Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ6P2OBFBjdaYYtKrV3A-9SXZZgUw8RpWuJ-V6FSIWbT0Iw-wZu5xI5JyTjvLIGVPVEPHBdKyfJ4iB2l_FB2zdybAqnizvM3OWPS2sy4Bn-3vwc1I1O0CXWMaGpV4UeI2CAf6XMPDX2yXehkwWHpa7ETUaM5XHs5f4WSxMNmwxhKIGk Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:06:10.953021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_f1061324903c42ed9f0306471a113b82", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/f1061324903c42ed9f0306471a113b82"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}] Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: be1f82d7a0a84fbebdd0dffef9f098ea Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:06:10.954666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '26ae0069-be67-4921-aca3-4cfae5214cf5'} Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:06:10.955662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:06:10.957225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:10.958515 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Returning 404 to user: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:06:10.958515 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1a90537b-a724-4040-b939-cb68d5a9a00d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 404 len: 111 microversion: 2.1 time: 0.030415 Oct 20 09:06:10.958515 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 399/805] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:10 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:06:10.963084 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4cc7bd8f-ab16-4cf0-b130-a102eba1ef2d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '.version_select at 0x7f5afb7b9b80>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:10.982147 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4cc7bd8f-ab16-4cf0-b130-a102eba1ef2d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-944100248" status: 202 len: 0 microversion: 2.1 time: 0.022793 Oct 20 09:06:10.983083 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 407/806] 10.222.0.36 () {58 vars in 1218 bytes} [Thu Oct 20 09:06:10 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-944100248 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:06:11.415261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2af8ba00-a9d3-4418-bc44-bb5f7c457768 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:11.416089 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2af8ba00-a9d3-4418-bc44-bb5f7c457768 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:11.422199 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2af8ba00-a9d3-4418-bc44-bb5f7c457768 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:11.422846 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2af8ba00-a9d3-4418-bc44-bb5f7c457768 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:11.423580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2af8ba00-a9d3-4418-bc44-bb5f7c457768 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:11.434769 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2af8ba00-a9d3-4418-bc44-bb5f7c457768 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:11.434769 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2af8ba00-a9d3-4418-bc44-bb5f7c457768 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:11.434769 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2af8ba00-a9d3-4418-bc44-bb5f7c457768 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:11.435240 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-676b2a6b-997b-473e-9b21-cd146af0fca3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:11.435762 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-676b2a6b-997b-473e-9b21-cd146af0fca3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:11.443333 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-676b2a6b-997b-473e-9b21-cd146af0fca3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:11.443728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-676b2a6b-997b-473e-9b21-cd146af0fca3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:11.444688 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-676b2a6b-997b-473e-9b21-cd146af0fca3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:11.445774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-676b2a6b-997b-473e-9b21-cd146af0fca3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:11.446757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-676b2a6b-997b-473e-9b21-cd146af0fca3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:11.447438 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-676b2a6b-997b-473e-9b21-cd146af0fca3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:11.476611 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-676b2a6b-997b-473e-9b21-cd146af0fca3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lazy-loading 'flavor' on Instance uuid 7facb789-628a-44aa-b0ec-7f6aee5617dd {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:06:11.590406 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2af8ba00-a9d3-4418-bc44-bb5f7c457768 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:11.602045 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2af8ba00-a9d3-4418-bc44-bb5f7c457768 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1364 microversion: 2.1 time: 0.193396 Oct 20 09:06:11.611770 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 400/807] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:11 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1364 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:11.665130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2765ac4e-6fc9-40d6-9eb8-0fd83485772a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'create', calling method: .version_select at 0x7f5afb974160>, body: {"keypair": {"name": "tempest-keypair-2003739164", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFixQiqBflzqCDgpI+UBxRM4XX2lutZlqFJ2+eXgcGI0xSQC6mCYQHf3OTuVx9YmtQHusnXJq6PT0oYO+jDMlYn/6A6AXWTffRNJPSwSR6Awc/8vBwonq24xM45B1GwG8w=="}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:11.685379 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-2765ac4e-6fc9-40d6-9eb8-0fd83485772a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Getting quotas for project f1061324903c42ed9f0306471a113b82. Resources: {'key_pairs'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:06:11.689836 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-2765ac4e-6fc9-40d6-9eb8-0fd83485772a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Getting quotas for user be1f82d7a0a84fbebdd0dffef9f098ea and project f1061324903c42ed9f0306471a113b82. Resources: {'key_pairs'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:06:11.703593 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-676b2a6b-997b-473e-9b21-cd146af0fca3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "POST /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/action" status: 202 len: 0 microversion: 2.20 time: 0.273363 Oct 20 09:06:11.704617 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 408/808] 10.222.0.36 () {62 vars in 1319 bytes} [Thu Oct 20 09:06:11 2022] POST /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/action => generated 0 bytes in 275 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 20 09:06:11.715839 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:11.716836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:11.717764 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2765ac4e-6fc9-40d6-9eb8-0fd83485772a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.059958 Oct 20 09:06:11.722653 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 401/809] 10.222.0.36 () {60 vars in 1159 bytes} [Thu Oct 20 09:06:11 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:11.726696 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:11.727290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:11.728047 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:11.728831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:11.729403 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:11.730458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:11.878482 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:11.886870 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:11.888302 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:11.889604 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-549a9f7b-0c1f-4702-9291-b83b0ba78c58 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.178341 Oct 20 09:06:11.890257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 409/810] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:11 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:12.625903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d110086e-2063-4e88-b1db-7e923a104da6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:12.626527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d110086e-2063-4e88-b1db-7e923a104da6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:12.632449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d110086e-2063-4e88-b1db-7e923a104da6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:12.632889 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d110086e-2063-4e88-b1db-7e923a104da6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:12.633309 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d110086e-2063-4e88-b1db-7e923a104da6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:12.633985 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d110086e-2063-4e88-b1db-7e923a104da6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:12.634428 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d110086e-2063-4e88-b1db-7e923a104da6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:12.634914 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d110086e-2063-4e88-b1db-7e923a104da6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:12.770087 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d110086e-2063-4e88-b1db-7e923a104da6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:12.777849 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d110086e-2063-4e88-b1db-7e923a104da6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1364 microversion: 2.1 time: 0.156296 Oct 20 09:06:12.778413 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 402/811] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:12 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1364 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:12.914848 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:12.915587 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:12.921928 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:12.922541 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:12.922866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:12.923552 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:12.924000 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:12.924413 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:12.971929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1330198855", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "26c5cd40-c195-42e5-baf1-db47fbf3d5b7"}], "security_groups": [{"name": "tempest-securitygroup--884303034"}], "key_name": "tempest-keypair-2003739164", "user_data": "}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:13.056794 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:13.065454 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:13.066935 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:13.068105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a179739a-82b4-40d0-a94f-71026eeb970c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.157072 Oct 20 09:06:13.068669 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 410/812] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:12 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:13.078847 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] validate_networks() for [('26c5cd40-c195-42e5-baf1-db47fbf3d5b7', None, None, None, None, None)] {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:06:13.530053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:06:13.530490 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:06:13.530959 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:06:13.531361 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:06:13.531768 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:06:13.540352 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:13.540828 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:13.541252 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:13.541961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:13.543027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:13.543453 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:13.559385 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Getting quotas for project f1061324903c42ed9f0306471a113b82. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:06:13.563249 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Getting quotas for user be1f82d7a0a84fbebdd0dffef9f098ea and project f1061324903c42ed9f0306471a113b82. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:06:13.573848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:06:13.576118 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:06:13.623414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2061f649-9b42-4dd3-b396-32e132130f72 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.655663 Oct 20 09:06:13.624069 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 403/813] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:06:12 2022] POST /compute/v2.1/servers => generated 401 bytes in 657 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 20 09:06:13.634998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a828dbea-8c5f-41db-8be7-780df9fc4663 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:13.635608 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a828dbea-8c5f-41db-8be7-780df9fc4663 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:13.689394 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a828dbea-8c5f-41db-8be7-780df9fc4663 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:13.696231 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a828dbea-8c5f-41db-8be7-780df9fc4663 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1238 microversion: 2.1 time: 0.064977 Oct 20 09:06:13.696793 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 411/814] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:13 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1238 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:13.800958 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2ab83a31-b68c-46d4-add0-af6c15c057bc tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:13.801561 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2ab83a31-b68c-46d4-add0-af6c15c057bc tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:13.808777 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2ab83a31-b68c-46d4-add0-af6c15c057bc tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:13.809254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2ab83a31-b68c-46d4-add0-af6c15c057bc tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:13.809681 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2ab83a31-b68c-46d4-add0-af6c15c057bc tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:13.810362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2ab83a31-b68c-46d4-add0-af6c15c057bc tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:13.810836 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2ab83a31-b68c-46d4-add0-af6c15c057bc tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:13.811254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2ab83a31-b68c-46d4-add0-af6c15c057bc tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:13.942861 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2ab83a31-b68c-46d4-add0-af6c15c057bc tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:13.951060 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2ab83a31-b68c-46d4-add0-af6c15c057bc tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1364 microversion: 2.1 time: 0.153860 Oct 20 09:06:13.951644 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 404/815] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:13 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1364 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:14.093823 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:14.094432 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:14.101555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:14.102031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:14.102455 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:14.103171 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:14.103615 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:14.104032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:14.229806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:14.238572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:14.240001 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:14.241154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3f820608-2731-4c15-be55-f54f396e812e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.152118 Oct 20 09:06:14.241710 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 412/816] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:14 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:14.719853 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-00385251-940e-42b3-9643-c4db38f5d93f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:14.720462 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-00385251-940e-42b3-9643-c4db38f5d93f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:14.726686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-00385251-940e-42b3-9643-c4db38f5d93f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:14.727170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-00385251-940e-42b3-9643-c4db38f5d93f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:14.727602 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-00385251-940e-42b3-9643-c4db38f5d93f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:14.728306 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-00385251-940e-42b3-9643-c4db38f5d93f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:14.728752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-00385251-940e-42b3-9643-c4db38f5d93f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:14.729169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-00385251-940e-42b3-9643-c4db38f5d93f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:14.791302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-00385251-940e-42b3-9643-c4db38f5d93f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:14.799456 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-00385251-940e-42b3-9643-c4db38f5d93f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1242 microversion: 2.1 time: 0.083391 Oct 20 09:06:14.800070 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 405/817] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:14 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1242 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:14.974634 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4ce482a8-d713-44c4-9830-52fe9825228e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:14.975251 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4ce482a8-d713-44c4-9830-52fe9825228e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:14.981917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4ce482a8-d713-44c4-9830-52fe9825228e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:14.982637 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4ce482a8-d713-44c4-9830-52fe9825228e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:14.983077 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4ce482a8-d713-44c4-9830-52fe9825228e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:14.983774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4ce482a8-d713-44c4-9830-52fe9825228e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:14.984228 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4ce482a8-d713-44c4-9830-52fe9825228e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:14.984653 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4ce482a8-d713-44c4-9830-52fe9825228e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:15.122904 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4ce482a8-d713-44c4-9830-52fe9825228e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:15.130920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4ce482a8-d713-44c4-9830-52fe9825228e tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1553 microversion: 2.1 time: 0.160049 Oct 20 09:06:15.131442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 413/818] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:14 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1553 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:15.154006 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0a2a94c4-4402-4592-9f93-d1fc0b1650b6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:15.154582 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0a2a94c4-4402-4592-9f93-d1fc0b1650b6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:15.161874 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2a94c4-4402-4592-9f93-d1fc0b1650b6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:15.162228 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2a94c4-4402-4592-9f93-d1fc0b1650b6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:15.162708 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2a94c4-4402-4592-9f93-d1fc0b1650b6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:15.163677 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2a94c4-4402-4592-9f93-d1fc0b1650b6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:15.164065 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2a94c4-4402-4592-9f93-d1fc0b1650b6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:15.164490 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0a2a94c4-4402-4592-9f93-d1fc0b1650b6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:15.248480 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0a2a94c4-4402-4592-9f93-d1fc0b1650b6 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.098780 Oct 20 09:06:15.249094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 406/819] 10.222.0.36 () {58 vars in 1275 bytes} [Thu Oct 20 09:06:15 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-interface => generated 284 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:15.265758 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:15.266365 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:15.280193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:15.280806 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:15.281310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:15.282077 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:15.282766 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:15.283257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:15.566270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:15.576263 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:15.577770 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:15.578990 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-14b9c7ab-86c3-49ab-b4ef-ffb6ead304c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.317745 Oct 20 09:06:15.579541 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 414/820] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:15 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:15.826907 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a6f9efcf-495e-4b1a-827a-fd026141533c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:15.827511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a6f9efcf-495e-4b1a-827a-fd026141533c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:15.833686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6f9efcf-495e-4b1a-827a-fd026141533c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:15.834176 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6f9efcf-495e-4b1a-827a-fd026141533c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:15.834650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6f9efcf-495e-4b1a-827a-fd026141533c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:15.835473 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6f9efcf-495e-4b1a-827a-fd026141533c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:15.835966 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6f9efcf-495e-4b1a-827a-fd026141533c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:15.836397 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a6f9efcf-495e-4b1a-827a-fd026141533c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:15.900857 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a6f9efcf-495e-4b1a-827a-fd026141533c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:15.909552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a6f9efcf-495e-4b1a-827a-fd026141533c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1290 microversion: 2.1 time: 0.086229 Oct 20 09:06:15.910420 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 407/821] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:15 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1290 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:16.341386 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-dd9dcf24-ab08-4899-ba68-e51c034908ee req-1d906a47-fe03-44ad-8aa8-3bd2c3eaedc1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c5dba102-6a02-40f4-a502-d1fc1cdab301", "tag": "a91a700e-d01e-4c84-8f53-4255f46188de"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:16.351132 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dd9dcf24-ab08-4899-ba68-e51c034908ee req-1d906a47-fe03-44ad-8aa8-3bd2c3eaedc1 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:16.351625 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dd9dcf24-ab08-4899-ba68-e51c034908ee req-1d906a47-fe03-44ad-8aa8-3bd2c3eaedc1 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:16.352081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dd9dcf24-ab08-4899-ba68-e51c034908ee req-1d906a47-fe03-44ad-8aa8-3bd2c3eaedc1 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:16.364350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-dd9dcf24-ab08-4899-ba68-e51c034908ee req-1d906a47-fe03-44ad-8aa8-3bd2c3eaedc1 service nova] Creating event network-changed:a91a700e-d01e-4c84-8f53-4255f46188de for instance c5dba102-6a02-40f4-a502-d1fc1cdab301 on hv-cinder-maste Oct 20 09:06:16.369496 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-dd9dcf24-ab08-4899-ba68-e51c034908ee req-1d906a47-fe03-44ad-8aa8-3bd2c3eaedc1 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.091662 Oct 20 09:06:16.373694 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 415/822] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:16.605033 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:16.606368 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:16.616875 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:16.619364 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:16.620138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:16.621916 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:16.622573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:16.623145 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:16.933726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-629d0345-f59c-4b78-83a0-e8469f974b2f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:16.934689 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-629d0345-f59c-4b78-83a0-e8469f974b2f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:16.941989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-629d0345-f59c-4b78-83a0-e8469f974b2f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:16.942733 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-629d0345-f59c-4b78-83a0-e8469f974b2f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:16.943470 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-629d0345-f59c-4b78-83a0-e8469f974b2f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:16.944391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-629d0345-f59c-4b78-83a0-e8469f974b2f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:16.945726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-629d0345-f59c-4b78-83a0-e8469f974b2f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:16.946392 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-629d0345-f59c-4b78-83a0-e8469f974b2f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:17.016462 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-629d0345-f59c-4b78-83a0-e8469f974b2f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:17.019364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:17.028861 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-629d0345-f59c-4b78-83a0-e8469f974b2f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1296 microversion: 2.1 time: 0.099155 Oct 20 09:06:17.030181 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 416/823] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:16 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1296 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:17.059942 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:17.069897 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:17.071767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:17.072955 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-047e5ae2-23d6-4cad-becd-17aa7c38034f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.471972 Oct 20 09:06:17.074071 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 408/824] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:16 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:18.054789 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d8f63f7e-54e4-47fe-b6c5-031897e017e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:18.055981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d8f63f7e-54e4-47fe-b6c5-031897e017e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:18.067261 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8f63f7e-54e4-47fe-b6c5-031897e017e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:18.067755 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8f63f7e-54e4-47fe-b6c5-031897e017e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:18.068188 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8f63f7e-54e4-47fe-b6c5-031897e017e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:18.068882 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8f63f7e-54e4-47fe-b6c5-031897e017e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:18.069329 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8f63f7e-54e4-47fe-b6c5-031897e017e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:18.069752 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d8f63f7e-54e4-47fe-b6c5-031897e017e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:18.099712 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:18.100392 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:18.113916 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:18.115102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:18.115597 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:18.116276 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:18.116723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:18.117323 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:18.231571 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d8f63f7e-54e4-47fe-b6c5-031897e017e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:18.239913 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d8f63f7e-54e4-47fe-b6c5-031897e017e4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1363 microversion: 2.1 time: 0.190452 Oct 20 09:06:18.240786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 417/825] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:18 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1363 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:18.362617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:18.376196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:18.377640 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:18.378879 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-74a6b50b-29ee-42d1-8a1b-332b33180417 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.283826 Oct 20 09:06:18.379437 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 409/826] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:18 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:18.444807 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-77489a87-feb7-4727-b19a-c539708e204d req-53c775c8-a63a-4fdd-8636-795466074fe3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "814ac79f-b35d-4aac-9b42-9f4ae8a83322", "tag": "df10e6d7-6494-4543-851c-4aa8a550df87"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:18.455050 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-77489a87-feb7-4727-b19a-c539708e204d req-53c775c8-a63a-4fdd-8636-795466074fe3 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:18.455746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-77489a87-feb7-4727-b19a-c539708e204d req-53c775c8-a63a-4fdd-8636-795466074fe3 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:18.456172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-77489a87-feb7-4727-b19a-c539708e204d req-53c775c8-a63a-4fdd-8636-795466074fe3 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:18.469002 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-77489a87-feb7-4727-b19a-c539708e204d req-53c775c8-a63a-4fdd-8636-795466074fe3 service nova] Creating event network-changed:df10e6d7-6494-4543-851c-4aa8a550df87 for instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 on hv-cinder-maste Oct 20 09:06:18.474061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-77489a87-feb7-4727-b19a-c539708e204d req-53c775c8-a63a-4fdd-8636-795466074fe3 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032903 Oct 20 09:06:18.474685 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 418/827] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:19.263248 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6239afad-f282-4a32-86f1-3c82b8c98e2a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:19.263860 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6239afad-f282-4a32-86f1-3c82b8c98e2a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:19.270154 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6239afad-f282-4a32-86f1-3c82b8c98e2a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:19.270679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6239afad-f282-4a32-86f1-3c82b8c98e2a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:19.271120 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6239afad-f282-4a32-86f1-3c82b8c98e2a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:19.271821 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6239afad-f282-4a32-86f1-3c82b8c98e2a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:19.275363 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6239afad-f282-4a32-86f1-3c82b8c98e2a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:19.275815 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6239afad-f282-4a32-86f1-3c82b8c98e2a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:19.408606 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:19.409501 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:19.415925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6239afad-f282-4a32-86f1-3c82b8c98e2a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:19.417885 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:19.418448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:19.418980 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:19.419788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:19.420347 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:19.421001 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:19.425788 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6239afad-f282-4a32-86f1-3c82b8c98e2a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1363 microversion: 2.1 time: 0.166246 Oct 20 09:06:19.426407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 410/828] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:19 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1363 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:19.570541 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:19.579310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:19.580760 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:19.581925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0bde73d2-696b-4d46-9827-ff5ec1ddae43 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.181092 Oct 20 09:06:19.582540 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 419/829] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:19 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:20.449213 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7cd88856-45dc-4b01-a10e-afcfad3e6100 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:20.450459 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7cd88856-45dc-4b01-a10e-afcfad3e6100 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:20.456815 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7cd88856-45dc-4b01-a10e-afcfad3e6100 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:20.457296 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7cd88856-45dc-4b01-a10e-afcfad3e6100 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:20.457727 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7cd88856-45dc-4b01-a10e-afcfad3e6100 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:20.458408 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7cd88856-45dc-4b01-a10e-afcfad3e6100 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:20.458895 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7cd88856-45dc-4b01-a10e-afcfad3e6100 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:20.459316 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7cd88856-45dc-4b01-a10e-afcfad3e6100 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:20.589069 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7cd88856-45dc-4b01-a10e-afcfad3e6100 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:20.597420 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7cd88856-45dc-4b01-a10e-afcfad3e6100 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1363 microversion: 2.1 time: 0.152044 Oct 20 09:06:20.598402 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 411/830] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:20 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1363 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:20.607426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:20.608022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:20.614653 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:20.615130 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:20.615558 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:20.616239 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:20.616686 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:20.617103 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:20.756685 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:20.765562 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:20.766996 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:20.768299 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-17343927-8238-4153-a02f-123acf742634 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.165171 Oct 20 09:06:20.779719 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 420/831] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:20 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:21.620919 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5be56d5c-4f5c-4ab3-b9f6-35f1b12aa03d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:21.622146 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5be56d5c-4f5c-4ab3-b9f6-35f1b12aa03d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:21.628514 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5be56d5c-4f5c-4ab3-b9f6-35f1b12aa03d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:21.628992 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5be56d5c-4f5c-4ab3-b9f6-35f1b12aa03d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:21.629429 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5be56d5c-4f5c-4ab3-b9f6-35f1b12aa03d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:21.630101 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5be56d5c-4f5c-4ab3-b9f6-35f1b12aa03d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:21.630574 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5be56d5c-4f5c-4ab3-b9f6-35f1b12aa03d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:21.630995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5be56d5c-4f5c-4ab3-b9f6-35f1b12aa03d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:21.755550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5be56d5c-4f5c-4ab3-b9f6-35f1b12aa03d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:21.763065 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5be56d5c-4f5c-4ab3-b9f6-35f1b12aa03d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1363 microversion: 2.1 time: 0.145977 Oct 20 09:06:21.763620 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 412/832] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:21 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1363 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:21.797276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:21.797887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:21.804468 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:21.804959 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:21.805398 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:21.806093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:21.806577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:21.807010 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:21.939664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:21.947871 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:21.949304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:21.950429 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-294c0a29-1e5b-4385-a01e-864dd2119447 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.157740 Oct 20 09:06:21.951007 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 421/833] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:21 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:22.785402 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b1a7aab9-440a-44c5-a1a4-ad7661c50969 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:22.786017 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b1a7aab9-440a-44c5-a1a4-ad7661c50969 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:22.793109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b1a7aab9-440a-44c5-a1a4-ad7661c50969 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:22.793588 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b1a7aab9-440a-44c5-a1a4-ad7661c50969 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:22.794023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b1a7aab9-440a-44c5-a1a4-ad7661c50969 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:22.794762 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b1a7aab9-440a-44c5-a1a4-ad7661c50969 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:22.795219 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b1a7aab9-440a-44c5-a1a4-ad7661c50969 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:22.795647 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b1a7aab9-440a-44c5-a1a4-ad7661c50969 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:22.859983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-dd1ca4e8-fa3d-4ecd-a3de-019873dbd494 req-3671dfa5-471a-4097-a247-b3d9ba2f64d6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "814ac79f-b35d-4aac-9b42-9f4ae8a83322", "name": "network-vif-plugged", "status": "completed", "tag": "df10e6d7-6494-4543-851c-4aa8a550df87"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:22.870222 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dd1ca4e8-fa3d-4ecd-a3de-019873dbd494 req-3671dfa5-471a-4097-a247-b3d9ba2f64d6 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:22.870769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dd1ca4e8-fa3d-4ecd-a3de-019873dbd494 req-3671dfa5-471a-4097-a247-b3d9ba2f64d6 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:22.871201 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dd1ca4e8-fa3d-4ecd-a3de-019873dbd494 req-3671dfa5-471a-4097-a247-b3d9ba2f64d6 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:22.882257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-dd1ca4e8-fa3d-4ecd-a3de-019873dbd494 req-3671dfa5-471a-4097-a247-b3d9ba2f64d6 service nova] Creating event network-vif-plugged:df10e6d7-6494-4543-851c-4aa8a550df87 for instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 on hv-cinder-maste Oct 20 09:06:22.887661 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-dd1ca4e8-fa3d-4ecd-a3de-019873dbd494 req-3671dfa5-471a-4097-a247-b3d9ba2f64d6 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096278 Oct 20 09:06:22.888258 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 422/834] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:06:22.931493 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b1a7aab9-440a-44c5-a1a4-ad7661c50969 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:22.939054 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b1a7aab9-440a-44c5-a1a4-ad7661c50969 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1363 microversion: 2.1 time: 0.157803 Oct 20 09:06:22.939615 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 413/835] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:22 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:22.976546 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:22.977242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:22.983898 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:22.984379 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:22.984814 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:22.985507 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:22.985968 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:22.986390 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:23.117513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:23.126460 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:23.127930 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:23.129108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7d3d1810-b6eb-4a4e-8680-1461a53e2b35 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.156871 Oct 20 09:06:23.129668 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 423/836] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:22 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:23.961983 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-85c80225-5ecc-436b-951a-342d903617fc tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:23.962627 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-85c80225-5ecc-436b-951a-342d903617fc tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:23.964591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:23.965804 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=10 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:23.972876 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85c80225-5ecc-436b-951a-342d903617fc tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:23.973369 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85c80225-5ecc-436b-951a-342d903617fc tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:23.973822 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85c80225-5ecc-436b-951a-342d903617fc tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:23.974522 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85c80225-5ecc-436b-951a-342d903617fc tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:23.975182 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85c80225-5ecc-436b-951a-342d903617fc tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:23.975606 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-85c80225-5ecc-436b-951a-342d903617fc tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:24.113528 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-85c80225-5ecc-436b-951a-342d903617fc tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:24.121084 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-85c80225-5ecc-436b-951a-342d903617fc tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1363 microversion: 2.1 time: 0.162936 Oct 20 09:06:24.122151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 414/837] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:23 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1363 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:24.153291 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:24.153891 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:24.155865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:24.161506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:24.161986 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:24.162417 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:24.163124 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:24.163573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:24.163995 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:24.288938 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:24.297575 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:24.299054 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:24.300212 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-202d8c6b-3fd5-41f4-a392-2ebd3c302676 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.150873 Oct 20 09:06:24.300788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 424/838] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:24 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:25.145172 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-051addc0-663d-499c-86ad-c64a2a397bf0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:25.145807 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-051addc0-663d-499c-86ad-c64a2a397bf0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:25.152140 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-051addc0-663d-499c-86ad-c64a2a397bf0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:25.153246 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-051addc0-663d-499c-86ad-c64a2a397bf0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:25.153676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-051addc0-663d-499c-86ad-c64a2a397bf0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:25.154350 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-051addc0-663d-499c-86ad-c64a2a397bf0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:25.154817 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-051addc0-663d-499c-86ad-c64a2a397bf0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:25.155235 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-051addc0-663d-499c-86ad-c64a2a397bf0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:25.170018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:25.284514 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-051addc0-663d-499c-86ad-c64a2a397bf0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:25.292228 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-051addc0-663d-499c-86ad-c64a2a397bf0 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1549 microversion: 2.1 time: 0.150836 Oct 20 09:06:25.292791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 415/839] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:25 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1549 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:25.315727 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2280af71-6163-40df-a968-d1bf7c48ceea tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:25.316238 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2280af71-6163-40df-a968-d1bf7c48ceea tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:25.322762 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2280af71-6163-40df-a968-d1bf7c48ceea tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:25.323255 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2280af71-6163-40df-a968-d1bf7c48ceea tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:25.323699 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2280af71-6163-40df-a968-d1bf7c48ceea tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:25.324394 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2280af71-6163-40df-a968-d1bf7c48ceea tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:25.324849 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2280af71-6163-40df-a968-d1bf7c48ceea tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:25.325238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:25.325381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2280af71-6163-40df-a968-d1bf7c48ceea tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:25.325951 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:25.331867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:25.332437 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:25.332939 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:25.333728 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:25.334275 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:25.334804 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:25.406154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2280af71-6163-40df-a968-d1bf7c48ceea tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.094216 Oct 20 09:06:25.406809 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 425/840] 10.222.0.36 () {58 vars in 1275 bytes} [Thu Oct 20 09:06:25 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:25.465493 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:25.473549 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:25.475015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:25.476167 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-dacd0e7a-5826-4343-8ee3-548d213f0cd4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.155116 Oct 20 09:06:25.476728 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 416/841] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:25 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:26.502495 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:26.503126 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:26.509113 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:26.509590 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:26.510019 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:26.510799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:26.511250 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:26.511664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:26.609572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-bf48e9af-2dd7-4ba7-8a46-446e804bd0a7 req-30b35127-79c5-4c90-9958-43b5034433b2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "814ac79f-b35d-4aac-9b42-9f4ae8a83322", "tag": "df10e6d7-6494-4543-851c-4aa8a550df87"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:26.619623 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-bf48e9af-2dd7-4ba7-8a46-446e804bd0a7 req-30b35127-79c5-4c90-9958-43b5034433b2 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:26.620287 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-bf48e9af-2dd7-4ba7-8a46-446e804bd0a7 req-30b35127-79c5-4c90-9958-43b5034433b2 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:26.620880 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-bf48e9af-2dd7-4ba7-8a46-446e804bd0a7 req-30b35127-79c5-4c90-9958-43b5034433b2 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:26.633066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-bf48e9af-2dd7-4ba7-8a46-446e804bd0a7 req-30b35127-79c5-4c90-9958-43b5034433b2 service nova] Creating event network-changed:df10e6d7-6494-4543-851c-4aa8a550df87 for instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 on hv-cinder-maste Oct 20 09:06:26.639929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-bf48e9af-2dd7-4ba7-8a46-446e804bd0a7 req-30b35127-79c5-4c90-9958-43b5034433b2 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033996 Oct 20 09:06:26.640534 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 417/842] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:06:26.673771 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:26.682707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:26.685066 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:26.686319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8598449f-94e0-4661-9c4a-c5e162504b0e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.188319 Oct 20 09:06:26.686919 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 426/843] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:26 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:27.712630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:27.713509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:27.721239 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:27.721706 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:27.722237 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:27.722954 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:27.723406 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:27.723860 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:27.856833 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:27.864910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:27.866343 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:27.867550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c5832380-b506-424e-9e49-6f8caf029fbe tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.159678 Oct 20 09:06:27.868111 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 418/844] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:27 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:28.893109 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:28.893707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:28.900251 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:28.900707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:28.901135 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:28.901853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:28.902281 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:28.902737 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:29.030871 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:29.038926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:29.040347 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:29.041487 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5249f74f-e48d-4014-aa29-4b97f9bda3f3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.152284 Oct 20 09:06:29.042045 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 427/845] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:28 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:29.083791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5ad13cb8-cdf6-4e4a-ba6f-8f115eddeb4f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:29.085988 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5ad13cb8-cdf6-4e4a-ba6f-8f115eddeb4f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:29.088376 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:29.094129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ad13cb8-cdf6-4e4a-ba6f-8f115eddeb4f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:29.094645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ad13cb8-cdf6-4e4a-ba6f-8f115eddeb4f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:29.095076 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ad13cb8-cdf6-4e4a-ba6f-8f115eddeb4f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:29.095752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ad13cb8-cdf6-4e4a-ba6f-8f115eddeb4f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:29.096194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ad13cb8-cdf6-4e4a-ba6f-8f115eddeb4f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:29.096612 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ad13cb8-cdf6-4e4a-ba6f-8f115eddeb4f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:29.170634 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5ad13cb8-cdf6-4e4a-ba6f-8f115eddeb4f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "POST /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/action" status: 202 len: 0 microversion: 2.1 time: 0.090392 Oct 20 09:06:29.171214 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 419/846] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:06:29 2022] POST /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/action => generated 0 bytes in 92 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:06:29.179234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-240d0c5f-ad35-4d01-9551-976ffa352289 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:29.179842 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-240d0c5f-ad35-4d01-9551-976ffa352289 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:29.186212 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-240d0c5f-ad35-4d01-9551-976ffa352289 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:29.186760 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-240d0c5f-ad35-4d01-9551-976ffa352289 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:29.187194 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-240d0c5f-ad35-4d01-9551-976ffa352289 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:29.187887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-240d0c5f-ad35-4d01-9551-976ffa352289 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:29.188506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-240d0c5f-ad35-4d01-9551-976ffa352289 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:29.188948 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-240d0c5f-ad35-4d01-9551-976ffa352289 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:29.351568 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-240d0c5f-ad35-4d01-9551-976ffa352289 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:29.361040 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-240d0c5f-ad35-4d01-9551-976ffa352289 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1753 microversion: 2.1 time: 0.185583 Oct 20 09:06:29.361817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 428/847] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:29 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1753 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:30.067623 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:30.068243 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:30.075160 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:30.075653 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:30.076121 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:30.076810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:30.077261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:30.077686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:30.204198 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:30.212155 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:30.213598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:30.215528 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c1c7377f-a85a-4516-847f-8cc4383d7aa3 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.151781 Oct 20 09:06:30.216098 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 420/848] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:30 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:30.385120 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d3e87e6e-f2e1-407c-abdf-13b51a724f62 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:30.385730 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d3e87e6e-f2e1-407c-abdf-13b51a724f62 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:30.392646 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d3e87e6e-f2e1-407c-abdf-13b51a724f62 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:30.393124 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d3e87e6e-f2e1-407c-abdf-13b51a724f62 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:30.393556 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d3e87e6e-f2e1-407c-abdf-13b51a724f62 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:30.394243 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d3e87e6e-f2e1-407c-abdf-13b51a724f62 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:30.394718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d3e87e6e-f2e1-407c-abdf-13b51a724f62 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:30.395139 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d3e87e6e-f2e1-407c-abdf-13b51a724f62 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:30.535728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d3e87e6e-f2e1-407c-abdf-13b51a724f62 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:30.544709 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d3e87e6e-f2e1-407c-abdf-13b51a724f62 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.163834 Oct 20 09:06:30.545280 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 429/849] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:30 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:31.241293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:31.241825 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:31.248225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:31.248705 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:31.249139 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:31.249810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:31.250256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:31.250817 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:31.378186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:31.386417 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:31.387986 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:31.389143 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-50033aba-b69c-4b95-9704-08fce052f395 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.151835 Oct 20 09:06:31.389698 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 421/850] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:31 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:31.568855 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bed9fade-4186-4b23-8986-4d8f09ce03a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:31.569373 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bed9fade-4186-4b23-8986-4d8f09ce03a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:31.575855 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bed9fade-4186-4b23-8986-4d8f09ce03a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:31.576337 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bed9fade-4186-4b23-8986-4d8f09ce03a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:31.576773 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bed9fade-4186-4b23-8986-4d8f09ce03a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:31.577461 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bed9fade-4186-4b23-8986-4d8f09ce03a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:31.577915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bed9fade-4186-4b23-8986-4d8f09ce03a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:31.578340 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bed9fade-4186-4b23-8986-4d8f09ce03a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:31.721663 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bed9fade-4186-4b23-8986-4d8f09ce03a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:31.729837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bed9fade-4186-4b23-8986-4d8f09ce03a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.165015 Oct 20 09:06:31.730926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 430/851] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:31 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:32.414225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:32.414861 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:32.421464 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:32.421955 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:32.422393 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:32.423151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:32.423864 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:32.424023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:32.554775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:32.563347 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:32.564771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:32.565905 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6ca4518e-de64-441f-ba10-91cbd0482ccc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.155770 Oct 20 09:06:32.566479 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 422/852] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:32 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:32.753403 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-23be27a9-3319-468e-a73f-24ae33f7d96d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:32.754042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-23be27a9-3319-468e-a73f-24ae33f7d96d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:32.760446 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-23be27a9-3319-468e-a73f-24ae33f7d96d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:32.760930 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-23be27a9-3319-468e-a73f-24ae33f7d96d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:32.761367 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-23be27a9-3319-468e-a73f-24ae33f7d96d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:32.762052 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-23be27a9-3319-468e-a73f-24ae33f7d96d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:32.762525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-23be27a9-3319-468e-a73f-24ae33f7d96d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:32.762952 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-23be27a9-3319-468e-a73f-24ae33f7d96d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:32.899978 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-23be27a9-3319-468e-a73f-24ae33f7d96d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:32.908887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-23be27a9-3319-468e-a73f-24ae33f7d96d tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.159386 Oct 20 09:06:32.909459 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 431/853] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:32 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:33.591556 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:33.592162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:33.598577 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:33.599060 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:33.599493 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:33.600171 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:33.600619 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:33.601040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:33.728206 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:33.736220 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:33.737692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:33.738875 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ce207e4e-5305-4e85-93cb-634fe8836b29 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.152087 Oct 20 09:06:33.739440 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 423/854] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:33 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:33.897564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bcc91c11-5569-452a-ba89-3009c549804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:33.898080 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bcc91c11-5569-452a-ba89-3009c549804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:33.904965 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bcc91c11-5569-452a-ba89-3009c549804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:33.905443 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bcc91c11-5569-452a-ba89-3009c549804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:33.905879 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bcc91c11-5569-452a-ba89-3009c549804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:33.906595 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bcc91c11-5569-452a-ba89-3009c549804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:33.907051 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bcc91c11-5569-452a-ba89-3009c549804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:33.907477 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bcc91c11-5569-452a-ba89-3009c549804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:33.932648 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6423e632-de75-4463-86fb-a75d66a96d4b tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:33.932851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bcc91c11-5569-452a-ba89-3009c549804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:06:33.933190 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6423e632-de75-4463-86fb-a75d66a96d4b tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:33.939045 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6423e632-de75-4463-86fb-a75d66a96d4b tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:33.939527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6423e632-de75-4463-86fb-a75d66a96d4b tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:33.939961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6423e632-de75-4463-86fb-a75d66a96d4b tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:33.940648 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6423e632-de75-4463-86fb-a75d66a96d4b tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:33.941097 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6423e632-de75-4463-86fb-a75d66a96d4b tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:33.941519 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6423e632-de75-4463-86fb-a75d66a96d4b tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:34.011339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bcc91c11-5569-452a-ba89-3009c549804d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "DELETE /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 204 len: 0 microversion: 2.1 time: 0.117207 Oct 20 09:06:34.012603 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 432/855] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:06:33 2022] DELETE /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:06:34.020959 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9facbe4e-c30f-4baa-9dcf-7f2557861b4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:34.021788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9facbe4e-c30f-4baa-9dcf-7f2557861b4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:34.028216 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9facbe4e-c30f-4baa-9dcf-7f2557861b4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:34.028717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9facbe4e-c30f-4baa-9dcf-7f2557861b4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:34.029136 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9facbe4e-c30f-4baa-9dcf-7f2557861b4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:34.029823 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9facbe4e-c30f-4baa-9dcf-7f2557861b4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:34.030270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9facbe4e-c30f-4baa-9dcf-7f2557861b4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:34.030723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9facbe4e-c30f-4baa-9dcf-7f2557861b4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:34.046754 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:34.108460 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6423e632-de75-4463-86fb-a75d66a96d4b tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:34.118000 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6423e632-de75-4463-86fb-a75d66a96d4b tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.189167 Oct 20 09:06:34.118591 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 424/856] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:33 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:34.172105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9facbe4e-c30f-4baa-9dcf-7f2557861b4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:34.180869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9facbe4e-c30f-4baa-9dcf-7f2557861b4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1579 microversion: 2.1 time: 0.164037 Oct 20 09:06:34.181442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 433/857] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:34 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1579 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:34.764753 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:34.765352 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:34.771803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:34.772285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:34.772716 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:34.773390 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:34.773837 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:34.774258 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:34.900304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:34.908345 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:34.909782 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:34.910961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-be62fc21-ddaf-4bc8-8f86-ce8035a09c9d tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.150589 Oct 20 09:06:34.911528 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 425/858] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:34 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:35.142345 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ac71a7cb-035c-48ce-997e-d50b0d3d2fe3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:35.142981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ac71a7cb-035c-48ce-997e-d50b0d3d2fe3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:35.149510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac71a7cb-035c-48ce-997e-d50b0d3d2fe3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:35.149992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac71a7cb-035c-48ce-997e-d50b0d3d2fe3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:35.150425 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac71a7cb-035c-48ce-997e-d50b0d3d2fe3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:35.151161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac71a7cb-035c-48ce-997e-d50b0d3d2fe3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:35.151608 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac71a7cb-035c-48ce-997e-d50b0d3d2fe3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:35.152027 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ac71a7cb-035c-48ce-997e-d50b0d3d2fe3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:35.205757 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cc443761-284d-44a5-a4f2-30baae1f6e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:35.206383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-cc443761-284d-44a5-a4f2-30baae1f6e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:35.212344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc443761-284d-44a5-a4f2-30baae1f6e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:35.212821 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc443761-284d-44a5-a4f2-30baae1f6e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:35.213251 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc443761-284d-44a5-a4f2-30baae1f6e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:35.213929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc443761-284d-44a5-a4f2-30baae1f6e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:35.214373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc443761-284d-44a5-a4f2-30baae1f6e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:35.214814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cc443761-284d-44a5-a4f2-30baae1f6e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:35.295309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ac71a7cb-035c-48ce-997e-d50b0d3d2fe3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:35.303698 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ac71a7cb-035c-48ce-997e-d50b0d3d2fe3 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.165154 Oct 20 09:06:35.304272 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 434/859] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:35 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:35.344772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-cc443761-284d-44a5-a4f2-30baae1f6e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:35.352800 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-cc443761-284d-44a5-a4f2-30baae1f6e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1579 microversion: 2.1 time: 0.151078 Oct 20 09:06:35.353363 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 426/860] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:35 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1579 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:35.936981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:35.937533 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:35.943538 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:35.944018 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:35.944480 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:35.945161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:35.945610 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:35.946036 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:36.079040 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:36.087246 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:36.088691 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:36.089869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-85e31f67-0c0b-4237-b715-bbcb170bf726 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1746 microversion: 2.20 time: 0.157461 Oct 20 09:06:36.090741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 435/861] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:35 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1746 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:36.327607 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8fbf69f6-e86c-4949-bb83-3e674754dbd2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:36.328148 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8fbf69f6-e86c-4949-bb83-3e674754dbd2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:36.334495 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8fbf69f6-e86c-4949-bb83-3e674754dbd2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:36.334982 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8fbf69f6-e86c-4949-bb83-3e674754dbd2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:36.335423 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8fbf69f6-e86c-4949-bb83-3e674754dbd2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:36.336142 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8fbf69f6-e86c-4949-bb83-3e674754dbd2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:36.336604 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8fbf69f6-e86c-4949-bb83-3e674754dbd2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:36.337020 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8fbf69f6-e86c-4949-bb83-3e674754dbd2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:36.375611 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5070c9a2-f791-4e6f-990b-5da531845382 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:36.376260 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5070c9a2-f791-4e6f-990b-5da531845382 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:36.384395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5070c9a2-f791-4e6f-990b-5da531845382 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:36.384891 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5070c9a2-f791-4e6f-990b-5da531845382 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:36.385334 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5070c9a2-f791-4e6f-990b-5da531845382 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:36.386070 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5070c9a2-f791-4e6f-990b-5da531845382 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:36.386607 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5070c9a2-f791-4e6f-990b-5da531845382 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:36.387036 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5070c9a2-f791-4e6f-990b-5da531845382 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:36.482978 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8fbf69f6-e86c-4949-bb83-3e674754dbd2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:36.491289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8fbf69f6-e86c-4949-bb83-3e674754dbd2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.167619 Oct 20 09:06:36.491865 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 427/862] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:36 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:36.523505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5070c9a2-f791-4e6f-990b-5da531845382 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:36.531281 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5070c9a2-f791-4e6f-990b-5da531845382 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1579 microversion: 2.1 time: 0.160092 Oct 20 09:06:36.531843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 436/863] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:36 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1579 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:37.115571 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:37.116175 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:37.122004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:37.122512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:37.123555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:37.124236 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:37.124684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:37.125103 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:37.252746 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:37.262498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:37.265235 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:37.267992 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-16c6ad80-d964-4d1a-bf12-0923cc9a7dbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1746 microversion: 2.20 time: 0.156211 Oct 20 09:06:37.268689 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 428/864] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:37 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1746 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:37.515792 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a3eebfbb-f71e-4ca3-ac00-82ab47249d68 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:37.516366 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a3eebfbb-f71e-4ca3-ac00-82ab47249d68 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:37.522796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a3eebfbb-f71e-4ca3-ac00-82ab47249d68 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:37.523253 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a3eebfbb-f71e-4ca3-ac00-82ab47249d68 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:37.523683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a3eebfbb-f71e-4ca3-ac00-82ab47249d68 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:37.524359 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a3eebfbb-f71e-4ca3-ac00-82ab47249d68 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:37.524803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a3eebfbb-f71e-4ca3-ac00-82ab47249d68 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:37.525218 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a3eebfbb-f71e-4ca3-ac00-82ab47249d68 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:37.554992 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d57438ed-fbd6-4bab-bb5b-4088a154fbf5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:37.555709 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d57438ed-fbd6-4bab-bb5b-4088a154fbf5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:37.562733 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d57438ed-fbd6-4bab-bb5b-4088a154fbf5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:37.563213 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d57438ed-fbd6-4bab-bb5b-4088a154fbf5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:37.563651 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d57438ed-fbd6-4bab-bb5b-4088a154fbf5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:37.564336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d57438ed-fbd6-4bab-bb5b-4088a154fbf5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:37.564786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d57438ed-fbd6-4bab-bb5b-4088a154fbf5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:37.565202 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d57438ed-fbd6-4bab-bb5b-4088a154fbf5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:37.665453 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a3eebfbb-f71e-4ca3-ac00-82ab47249d68 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:37.675024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a3eebfbb-f71e-4ca3-ac00-82ab47249d68 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.163531 Oct 20 09:06:37.675875 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 437/865] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:37 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:37.830609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-4cf81112-a392-4427-94ad-a02ac98eb25f req-ab9f4299-e316-4b8a-acfa-178de34a0a28 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7facb789-628a-44aa-b0ec-7f6aee5617dd", "name": "network-vif-plugged", "status": "completed", "tag": "f17302af-3ee4-46d2-a6fe-a973b183f4e4"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:37.843480 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-4cf81112-a392-4427-94ad-a02ac98eb25f req-ab9f4299-e316-4b8a-acfa-178de34a0a28 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:37.844058 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-4cf81112-a392-4427-94ad-a02ac98eb25f req-ab9f4299-e316-4b8a-acfa-178de34a0a28 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:37.845141 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-4cf81112-a392-4427-94ad-a02ac98eb25f req-ab9f4299-e316-4b8a-acfa-178de34a0a28 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:37.857843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-4cf81112-a392-4427-94ad-a02ac98eb25f req-ab9f4299-e316-4b8a-acfa-178de34a0a28 service nova] Creating event network-vif-plugged:f17302af-3ee4-46d2-a6fe-a973b183f4e4 for instance 7facb789-628a-44aa-b0ec-7f6aee5617dd on hv-cinder-maste Oct 20 09:06:37.863378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-4cf81112-a392-4427-94ad-a02ac98eb25f req-ab9f4299-e316-4b8a-acfa-178de34a0a28 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036728 Oct 20 09:06:37.864590 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 438/866] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:37.932265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d57438ed-fbd6-4bab-bb5b-4088a154fbf5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:37.940537 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d57438ed-fbd6-4bab-bb5b-4088a154fbf5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1579 microversion: 2.1 time: 0.389315 Oct 20 09:06:37.941107 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 429/867] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:37 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1579 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:38.298689 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:38.299388 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:38.305669 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:38.306354 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:38.306815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:38.307507 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:38.307959 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:38.308383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:38.463598 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:38.473420 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:38.475209 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:38.476756 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3b086151-e2c0-4ec3-a5d6-5848e636cf32 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1746 microversion: 2.20 time: 0.182582 Oct 20 09:06:38.477531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 439/868] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:38 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1746 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:38.700500 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2917b596-c086-41dc-ab05-f589045a0ee9 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:38.701679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2917b596-c086-41dc-ab05-f589045a0ee9 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:38.708345 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2917b596-c086-41dc-ab05-f589045a0ee9 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:38.708833 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2917b596-c086-41dc-ab05-f589045a0ee9 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:38.709272 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2917b596-c086-41dc-ab05-f589045a0ee9 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:38.709967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2917b596-c086-41dc-ab05-f589045a0ee9 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:38.710514 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2917b596-c086-41dc-ab05-f589045a0ee9 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:38.710967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2917b596-c086-41dc-ab05-f589045a0ee9 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:38.834200 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-c0d771c9-02fd-4959-abc0-9dfbab25376d req-111f3139-b878-45af-b563-c04d50e18cb8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b1d0b97-f53e-40dc-895f-48be8de838ee", "name": "network-vif-deleted", "tag": "ea6a461f-d2ae-49cf-a87f-19296d8271f9"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:38.847723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-c0d771c9-02fd-4959-abc0-9dfbab25376d req-111f3139-b878-45af-b563-c04d50e18cb8 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:38.848347 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-c0d771c9-02fd-4959-abc0-9dfbab25376d req-111f3139-b878-45af-b563-c04d50e18cb8 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:38.848831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-c0d771c9-02fd-4959-abc0-9dfbab25376d req-111f3139-b878-45af-b563-c04d50e18cb8 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:38.885195 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-c0d771c9-02fd-4959-abc0-9dfbab25376d req-111f3139-b878-45af-b563-c04d50e18cb8 service nova] Creating event network-vif-deleted:ea6a461f-d2ae-49cf-a87f-19296d8271f9 for instance 4b1d0b97-f53e-40dc-895f-48be8de838ee on hv-cinder-maste Oct 20 09:06:38.896113 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-c0d771c9-02fd-4959-abc0-9dfbab25376d req-111f3139-b878-45af-b563-c04d50e18cb8 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070830 Oct 20 09:06:38.897852 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 440/869] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:06:38.919674 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2917b596-c086-41dc-ab05-f589045a0ee9 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:38.929130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2917b596-c086-41dc-ab05-f589045a0ee9 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.233129 Oct 20 09:06:38.929723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 430/870] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:38 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:38.966165 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-dbe45313-d7ff-4b78-a857-f34002fbb43a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:38.967498 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-dbe45313-d7ff-4b78-a857-f34002fbb43a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:38.975583 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbe45313-d7ff-4b78-a857-f34002fbb43a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:38.976258 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbe45313-d7ff-4b78-a857-f34002fbb43a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:38.976865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbe45313-d7ff-4b78-a857-f34002fbb43a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:38.977723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbe45313-d7ff-4b78-a857-f34002fbb43a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:38.978322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbe45313-d7ff-4b78-a857-f34002fbb43a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:38.978963 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dbe45313-d7ff-4b78-a857-f34002fbb43a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:39.064386 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-dbe45313-d7ff-4b78-a857-f34002fbb43a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:39.074767 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-dbe45313-d7ff-4b78-a857-f34002fbb43a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 200 len: 1226 microversion: 2.1 time: 0.114078 Oct 20 09:06:39.075993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 441/871] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:38 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 1226 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:39.503787 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:39.504405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:39.511653 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:39.512143 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:39.512573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:39.513435 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:39.514066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:39.514502 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:39.672628 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:39.675041 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:39.684672 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:39.692023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:39.693202 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8b381be6-b5d4-4d64-9359-32d7dfeacb8f tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1746 microversion: 2.20 time: 0.193744 Oct 20 09:06:39.693992 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 431/872] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:39 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1746 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:39.953547 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c9aac8bf-3b0d-4f92-83fa-8d8f9cca8b6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:39.954159 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c9aac8bf-3b0d-4f92-83fa-8d8f9cca8b6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:39.961370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c9aac8bf-3b0d-4f92-83fa-8d8f9cca8b6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:39.961891 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c9aac8bf-3b0d-4f92-83fa-8d8f9cca8b6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:39.962324 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c9aac8bf-3b0d-4f92-83fa-8d8f9cca8b6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:39.963031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c9aac8bf-3b0d-4f92-83fa-8d8f9cca8b6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:39.963477 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c9aac8bf-3b0d-4f92-83fa-8d8f9cca8b6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:39.963896 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c9aac8bf-3b0d-4f92-83fa-8d8f9cca8b6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:40.098630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:40.099249 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:40.106650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:40.107668 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:40.108115 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:40.108814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:40.109269 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:40.109702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:40.123851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-c9aac8bf-3b0d-4f92-83fa-8d8f9cca8b6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:40.124747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] HTTP exception thrown: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Request method failure captured: Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee HTTP/1.1 Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ7eeucTVD7DXQxu3n1HjFWasd0RlEnVQ7oXQ_5m8nyMDw2lV0skX3kJnuhAmasbAkMuuiafaZjUtOq0VyoEu1ww7D6-6g-C3iagvZtTiBjAkt4-LmcRo8IFuiY3H5D3qhIk_XxHFiP0oYVzcbk-uqb10V9y86uVAD2IxN7EFc4Z38E Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 98827dbef155480b9f26792b70014b4b Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:06:40.126650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/98827dbef155480b9f26792b70014b4b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_98827dbef155480b9f26792b70014b4b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/98827dbef155480b9f26792b70014b4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/98827dbef155480b9f26792b70014b4b"}]}] Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 98827dbef155480b9f26792b70014b4b Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-TestVolumeBootPattern-1573908492-project Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 65ca2eae6db24bb094afaacdfb5c1980 Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-TestVolumeBootPattern-1573908492-project Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:06:40.128336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '4b1d0b97-f53e-40dc-895f-48be8de838ee'} Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:06:40.129344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:06:40.131342 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:06:40.132609 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Returning 404 to user: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:06:40.132609 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a7e3a445-64d4-4c42-8264-55b28a955633 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 404 len: 111 microversion: 2.1 time: 0.033934 Oct 20 09:06:40.132609 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 432/873] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:40 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 20 09:06:40.146531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c9aac8bf-3b0d-4f92-83fa-8d8f9cca8b6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.197264 Oct 20 09:06:40.153319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 442/874] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:39 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:40.229633 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-331645803", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1088348766", "security_groups": [{"name": "tempest-secgroup-smoke-1673307534"}], "block_device_mapping_v2": [{"uuid": "3e371462-e17e-4d91-b898-d08f40a65134", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "d2d4ae81-03f2-49c0-8ca3-a270479acf30"}]}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:40.350127 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] validate_networks() for [('d2d4ae81-03f2-49c0-8ca3-a270479acf30', None, None, None, None, None)] {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:06:40.721320 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:40.721940 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:40.739255 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=84,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:40.743007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:06:40.743227 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:40.743439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:06:40.743724 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:40.744092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:06:40.744219 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:40.744501 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:06:40.744925 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:40.745096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:06:40.745416 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:40.745809 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:40.756785 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:40.757266 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:40.757696 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:40.758409 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:40.759018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:40.759627 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:40.790877 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Getting quotas for project 98827dbef155480b9f26792b70014b4b. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:06:40.796632 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Getting quotas for user 65ca2eae6db24bb094afaacdfb5c1980 and project 98827dbef155480b9f26792b70014b4b. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:06:40.874171 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:06:40.876720 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='3e371462-e17e-4d91-b898-d08f40a65134',volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:06:40.961154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:40.973466 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:40.975355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:40.976945 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7d0befe1-8fd3-4e05-a3e8-50a65675eb91 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1741 microversion: 2.20 time: 0.261569 Oct 20 09:06:40.977795 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 443/875] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:40 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1741 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:41.068501 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-563e8844-b8a4-4201-bbec-580154dfbbc9 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.842513 Oct 20 09:06:41.069174 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 433/876] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:06:40 2022] POST /compute/v2.1/servers => generated 406 bytes in 844 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 20 09:06:41.082533 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d5ee6b61-14ad-4174-b314-fbff3cc89693 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:41.094804 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d5ee6b61-14ad-4174-b314-fbff3cc89693 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:41.170770 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1e79b1a5-2595-4cca-88a1-bfe0629567d2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:41.171371 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1e79b1a5-2595-4cca-88a1-bfe0629567d2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:41.187147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e79b1a5-2595-4cca-88a1-bfe0629567d2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:41.187838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e79b1a5-2595-4cca-88a1-bfe0629567d2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:41.188608 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e79b1a5-2595-4cca-88a1-bfe0629567d2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:41.189462 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e79b1a5-2595-4cca-88a1-bfe0629567d2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:41.190072 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e79b1a5-2595-4cca-88a1-bfe0629567d2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:41.190682 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1e79b1a5-2595-4cca-88a1-bfe0629567d2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:41.195962 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d5ee6b61-14ad-4174-b314-fbff3cc89693 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:41.211302 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d5ee6b61-14ad-4174-b314-fbff3cc89693 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1094 microversion: 2.1 time: 0.131557 Oct 20 09:06:41.213107 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 444/877] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:41 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1094 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:41.512479 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1e79b1a5-2595-4cca-88a1-bfe0629567d2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:41.521783 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1e79b1a5-2595-4cca-88a1-bfe0629567d2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.355412 Oct 20 09:06:41.523005 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 434/878] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:41 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:42.004773 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:42.006048 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:42.013043 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:42.014441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:42.015159 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:42.016097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:42.016783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:42.017564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:42.193887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:42.206081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:42.208347 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:42.209853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0d943012-4f77-4e50-9ed2-428b5992202a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1741 microversion: 2.20 time: 0.209378 Oct 20 09:06:42.225922 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 445/879] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:41 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1741 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:42.240060 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d204ce5d-7fe8-44bd-96fa-94dc5d807d60 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:42.241432 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d204ce5d-7fe8-44bd-96fa-94dc5d807d60 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:42.250985 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d204ce5d-7fe8-44bd-96fa-94dc5d807d60 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:42.251473 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d204ce5d-7fe8-44bd-96fa-94dc5d807d60 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:42.252246 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d204ce5d-7fe8-44bd-96fa-94dc5d807d60 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:42.253373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d204ce5d-7fe8-44bd-96fa-94dc5d807d60 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:42.253829 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d204ce5d-7fe8-44bd-96fa-94dc5d807d60 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:42.255429 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d204ce5d-7fe8-44bd-96fa-94dc5d807d60 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:42.332167 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d204ce5d-7fe8-44bd-96fa-94dc5d807d60 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:42.340983 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d204ce5d-7fe8-44bd-96fa-94dc5d807d60 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1192 microversion: 2.1 time: 0.105720 Oct 20 09:06:42.342538 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 435/880] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:42 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1192 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:42.546272 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1888cce5-d1b2-4e32-b8e1-3e13f9f210e7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:42.546809 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1888cce5-d1b2-4e32-b8e1-3e13f9f210e7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:42.552939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1888cce5-d1b2-4e32-b8e1-3e13f9f210e7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:42.553802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1888cce5-d1b2-4e32-b8e1-3e13f9f210e7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:42.554316 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1888cce5-d1b2-4e32-b8e1-3e13f9f210e7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:42.555152 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1888cce5-d1b2-4e32-b8e1-3e13f9f210e7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:42.555688 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1888cce5-d1b2-4e32-b8e1-3e13f9f210e7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:42.556194 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1888cce5-d1b2-4e32-b8e1-3e13f9f210e7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:42.729865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1888cce5-d1b2-4e32-b8e1-3e13f9f210e7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:42.740723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1888cce5-d1b2-4e32-b8e1-3e13f9f210e7 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.198247 Oct 20 09:06:42.754495 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 446/881] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:42 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:43.251692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:43.253058 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:43.260155 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:43.260639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:43.261365 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:43.262102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:43.262737 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:43.263688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:43.366763 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-947ae76d-28ef-452d-8523-4b05353f2e64 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:43.368122 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-947ae76d-28ef-452d-8523-4b05353f2e64 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:43.380452 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-947ae76d-28ef-452d-8523-4b05353f2e64 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:43.381340 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-947ae76d-28ef-452d-8523-4b05353f2e64 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:43.382882 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-947ae76d-28ef-452d-8523-4b05353f2e64 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:43.383839 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-947ae76d-28ef-452d-8523-4b05353f2e64 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:43.384751 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-947ae76d-28ef-452d-8523-4b05353f2e64 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:43.389686 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-947ae76d-28ef-452d-8523-4b05353f2e64 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:43.467590 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-947ae76d-28ef-452d-8523-4b05353f2e64 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:43.477203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-947ae76d-28ef-452d-8523-4b05353f2e64 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1210 microversion: 2.1 time: 0.114307 Oct 20 09:06:43.478543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 447/882] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:43 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1210 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:43.520785 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:43.534144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:43.535650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:43.536975 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-76ac432a-9028-43f6-9260-1bc5affc2bb2 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1741 microversion: 2.20 time: 0.289040 Oct 20 09:06:43.538038 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 436/883] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:43 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1741 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:43.766195 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a5135692-220d-4da7-9103-dea8536d9c1e tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:43.767235 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a5135692-220d-4da7-9103-dea8536d9c1e tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:43.776914 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5135692-220d-4da7-9103-dea8536d9c1e tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:43.777787 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5135692-220d-4da7-9103-dea8536d9c1e tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:43.778499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5135692-220d-4da7-9103-dea8536d9c1e tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:43.779981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5135692-220d-4da7-9103-dea8536d9c1e tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:43.780536 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5135692-220d-4da7-9103-dea8536d9c1e tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:43.781221 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a5135692-220d-4da7-9103-dea8536d9c1e tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:43.974670 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a5135692-220d-4da7-9103-dea8536d9c1e tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:43.984832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a5135692-220d-4da7-9103-dea8536d9c1e tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.223528 Oct 20 09:06:43.985764 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 448/884] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:43 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:44.502690 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a5c20472-888c-463a-9311-07ad6b93ce18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:44.503243 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a5c20472-888c-463a-9311-07ad6b93ce18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:44.509590 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5c20472-888c-463a-9311-07ad6b93ce18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:44.510211 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5c20472-888c-463a-9311-07ad6b93ce18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:44.510879 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5c20472-888c-463a-9311-07ad6b93ce18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:44.511739 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5c20472-888c-463a-9311-07ad6b93ce18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:44.512355 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5c20472-888c-463a-9311-07ad6b93ce18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:44.512943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a5c20472-888c-463a-9311-07ad6b93ce18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:44.563663 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:44.564257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:44.577347 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:44.578123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:44.578773 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:44.579480 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:44.580105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:44.580526 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:44.672883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a5c20472-888c-463a-9311-07ad6b93ce18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:44.681439 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a5c20472-888c-463a-9311-07ad6b93ce18 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1278 microversion: 2.1 time: 0.184131 Oct 20 09:06:44.682999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 437/885] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:44 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1278 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:44.735568 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:44.745591 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:44.747299 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:44.748676 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9884fe15-339e-489c-b8fd-8cd0190222bb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1741 microversion: 2.20 time: 0.189525 Oct 20 09:06:44.758878 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 449/886] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:44 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1741 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:45.012300 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ca8dfa1b-1915-41a9-b456-5d73837c8638 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:45.012827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ca8dfa1b-1915-41a9-b456-5d73837c8638 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:45.019271 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca8dfa1b-1915-41a9-b456-5d73837c8638 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:45.019864 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca8dfa1b-1915-41a9-b456-5d73837c8638 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:45.020508 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca8dfa1b-1915-41a9-b456-5d73837c8638 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:45.021302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca8dfa1b-1915-41a9-b456-5d73837c8638 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:45.021884 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca8dfa1b-1915-41a9-b456-5d73837c8638 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:45.022375 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ca8dfa1b-1915-41a9-b456-5d73837c8638 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:45.176628 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ca8dfa1b-1915-41a9-b456-5d73837c8638 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:45.186015 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ca8dfa1b-1915-41a9-b456-5d73837c8638 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.177027 Oct 20 09:06:45.186617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 438/887] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:45 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:45.705332 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-29079dfd-eb9e-4a2c-9922-507adc55d9ea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:45.706172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-29079dfd-eb9e-4a2c-9922-507adc55d9ea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:45.712908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29079dfd-eb9e-4a2c-9922-507adc55d9ea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:45.713618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29079dfd-eb9e-4a2c-9922-507adc55d9ea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:45.714067 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29079dfd-eb9e-4a2c-9922-507adc55d9ea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:45.714887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29079dfd-eb9e-4a2c-9922-507adc55d9ea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:45.715838 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29079dfd-eb9e-4a2c-9922-507adc55d9ea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:45.717124 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-29079dfd-eb9e-4a2c-9922-507adc55d9ea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:45.777023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:45.779313 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:45.787522 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:45.789169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:45.793461 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:45.794637 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:45.796652 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:45.797292 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:45.869327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-29079dfd-eb9e-4a2c-9922-507adc55d9ea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:45.879995 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-29079dfd-eb9e-4a2c-9922-507adc55d9ea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1278 microversion: 2.1 time: 0.178407 Oct 20 09:06:45.881381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 450/888] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:45 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1278 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:46.048937 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:46.057647 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:46.059125 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:46.060414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-496c398b-758d-48f5-aea0-1967af20e793 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1741 microversion: 2.20 time: 0.287169 Oct 20 09:06:46.084233 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 439/889] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:45 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1741 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:46.211093 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b44912a1-3c70-4918-80cc-dca346f8cb95 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:46.211770 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b44912a1-3c70-4918-80cc-dca346f8cb95 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:46.220857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b44912a1-3c70-4918-80cc-dca346f8cb95 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:46.221334 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b44912a1-3c70-4918-80cc-dca346f8cb95 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:46.221797 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b44912a1-3c70-4918-80cc-dca346f8cb95 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:46.222692 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b44912a1-3c70-4918-80cc-dca346f8cb95 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:46.223455 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b44912a1-3c70-4918-80cc-dca346f8cb95 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:46.224387 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b44912a1-3c70-4918-80cc-dca346f8cb95 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:46.397423 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b44912a1-3c70-4918-80cc-dca346f8cb95 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:46.410144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b44912a1-3c70-4918-80cc-dca346f8cb95 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.204586 Oct 20 09:06:46.411269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 451/890] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:46 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:46.762016 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-4e85fbb7-b15c-4b39-b367-038d3a6398f4 req-d22447d2-aad4-423c-ae3a-757831b26043 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9c52d13f-7238-490c-a0cd-c1d2fff9afb0", "tag": "d5aa2d4a-df18-490f-b5c0-9074a308160e"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:46.777888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-4e85fbb7-b15c-4b39-b367-038d3a6398f4 req-d22447d2-aad4-423c-ae3a-757831b26043 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:46.788174 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-4e85fbb7-b15c-4b39-b367-038d3a6398f4 req-d22447d2-aad4-423c-ae3a-757831b26043 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:46.788717 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-4e85fbb7-b15c-4b39-b367-038d3a6398f4 req-d22447d2-aad4-423c-ae3a-757831b26043 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:46.804970 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-4e85fbb7-b15c-4b39-b367-038d3a6398f4 req-d22447d2-aad4-423c-ae3a-757831b26043 service nova] Creating event network-changed:d5aa2d4a-df18-490f-b5c0-9074a308160e for instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 on hv-cinder-maste Oct 20 09:06:46.811231 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-4e85fbb7-b15c-4b39-b367-038d3a6398f4 req-d22447d2-aad4-423c-ae3a-757831b26043 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056593 Oct 20 09:06:46.812236 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 440/891] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:06:46.911572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-775dbc59-324e-45ad-9fff-306d1b617415 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:46.912170 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-775dbc59-324e-45ad-9fff-306d1b617415 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:46.918713 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-775dbc59-324e-45ad-9fff-306d1b617415 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:46.919196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-775dbc59-324e-45ad-9fff-306d1b617415 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:46.919688 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-775dbc59-324e-45ad-9fff-306d1b617415 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:46.920391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-775dbc59-324e-45ad-9fff-306d1b617415 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:46.920832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-775dbc59-324e-45ad-9fff-306d1b617415 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:46.921250 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-775dbc59-324e-45ad-9fff-306d1b617415 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:46.938729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=100 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:47.080061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-775dbc59-324e-45ad-9fff-306d1b617415 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:47.093660 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:47.094367 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:47.104734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-775dbc59-324e-45ad-9fff-306d1b617415 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.196131 Oct 20 09:06:47.104734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 452/892] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:46 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:47.105797 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:47.106557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:47.108090 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:47.109387 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:47.110003 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:47.110613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:47.364892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:47.376040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:47.377929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:47.379407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f83a426b-0870-45bc-a6dc-9deb8ba45057 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1741 microversion: 2.20 time: 0.290904 Oct 20 09:06:47.380214 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 441/893] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:47 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1741 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:47.433700 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-28281585-e3e7-482c-ad74-017e0d0cc57c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:47.434802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-28281585-e3e7-482c-ad74-017e0d0cc57c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:47.443413 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28281585-e3e7-482c-ad74-017e0d0cc57c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:47.444440 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28281585-e3e7-482c-ad74-017e0d0cc57c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:47.445144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28281585-e3e7-482c-ad74-017e0d0cc57c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:47.446241 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28281585-e3e7-482c-ad74-017e0d0cc57c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:47.446926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28281585-e3e7-482c-ad74-017e0d0cc57c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:47.447521 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-28281585-e3e7-482c-ad74-017e0d0cc57c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:47.679403 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-28281585-e3e7-482c-ad74-017e0d0cc57c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:47.689331 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-28281585-e3e7-482c-ad74-017e0d0cc57c tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1758 microversion: 2.1 time: 0.259963 Oct 20 09:06:47.690566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 453/894] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:47 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1758 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:06:48.128010 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c926c0ca-e6b6-47a8-84c8-6addc1003428 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:48.129351 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c926c0ca-e6b6-47a8-84c8-6addc1003428 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:48.147469 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c926c0ca-e6b6-47a8-84c8-6addc1003428 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:48.148297 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c926c0ca-e6b6-47a8-84c8-6addc1003428 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:48.149038 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c926c0ca-e6b6-47a8-84c8-6addc1003428 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:48.151051 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c926c0ca-e6b6-47a8-84c8-6addc1003428 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:48.151796 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c926c0ca-e6b6-47a8-84c8-6addc1003428 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:48.152547 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c926c0ca-e6b6-47a8-84c8-6addc1003428 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:48.409448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:48.410865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:48.420013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:48.420745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:48.421171 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:48.422137 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:48.422752 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:48.423566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:48.460771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c926c0ca-e6b6-47a8-84c8-6addc1003428 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:48.469779 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c926c0ca-e6b6-47a8-84c8-6addc1003428 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.345844 Oct 20 09:06:48.470341 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 442/895] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:48 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:48.596393 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:48.607947 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:48.609751 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:48.611963 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-72eea223-932b-457a-b2ec-8a40619ae982 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1741 microversion: 2.20 time: 0.206579 Oct 20 09:06:48.613020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 454/896] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:48 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1741 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:48.714915 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-07debfb0-653f-49f1-9379-5e2a1f3436f6 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:48.716011 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-07debfb0-653f-49f1-9379-5e2a1f3436f6 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:48.724912 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07debfb0-653f-49f1-9379-5e2a1f3436f6 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:48.725889 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07debfb0-653f-49f1-9379-5e2a1f3436f6 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:48.726325 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07debfb0-653f-49f1-9379-5e2a1f3436f6 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:48.727211 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07debfb0-653f-49f1-9379-5e2a1f3436f6 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:48.727895 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07debfb0-653f-49f1-9379-5e2a1f3436f6 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:48.728467 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-07debfb0-653f-49f1-9379-5e2a1f3436f6 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:48.928036 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-07debfb0-653f-49f1-9379-5e2a1f3436f6 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:48.940725 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-07debfb0-653f-49f1-9379-5e2a1f3436f6 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1761 microversion: 2.1 time: 0.230565 Oct 20 09:06:48.955006 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 443/897] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:48 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1761 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:49.493859 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-79dc9c23-db51-4545-a460-c1e03824dc21 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:49.495161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-79dc9c23-db51-4545-a460-c1e03824dc21 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:49.502580 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-79dc9c23-db51-4545-a460-c1e03824dc21 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:49.503290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-79dc9c23-db51-4545-a460-c1e03824dc21 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:49.503920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-79dc9c23-db51-4545-a460-c1e03824dc21 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:49.504779 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-79dc9c23-db51-4545-a460-c1e03824dc21 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:49.505404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-79dc9c23-db51-4545-a460-c1e03824dc21 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:49.505827 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-79dc9c23-db51-4545-a460-c1e03824dc21 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:49.642888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:49.643593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:49.653272 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:49.653819 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:49.654255 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:49.654991 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:49.655437 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:49.656538 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:49.707122 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-79dc9c23-db51-4545-a460-c1e03824dc21 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:49.719477 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-79dc9c23-db51-4545-a460-c1e03824dc21 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.230830 Oct 20 09:06:49.721199 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 455/898] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:49 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:49.938917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:49.952388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:49.953945 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:49.958087 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8f632e89-8fba-4274-9aa8-57eefcf7d58a tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1741 microversion: 2.20 time: 0.321678 Oct 20 09:06:49.958686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 444/899] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:49 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1741 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:50.746326 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5f0b98aa-2bcf-4e85-8fbe-0fe6b2082913 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:50.747068 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5f0b98aa-2bcf-4e85-8fbe-0fe6b2082913 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:50.756977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f0b98aa-2bcf-4e85-8fbe-0fe6b2082913 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:50.757524 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f0b98aa-2bcf-4e85-8fbe-0fe6b2082913 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:50.758333 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f0b98aa-2bcf-4e85-8fbe-0fe6b2082913 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:50.759408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f0b98aa-2bcf-4e85-8fbe-0fe6b2082913 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:50.760012 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f0b98aa-2bcf-4e85-8fbe-0fe6b2082913 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:50.760645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5f0b98aa-2bcf-4e85-8fbe-0fe6b2082913 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:50.927958 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5f0b98aa-2bcf-4e85-8fbe-0fe6b2082913 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:50.940389 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5f0b98aa-2bcf-4e85-8fbe-0fe6b2082913 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.200232 Oct 20 09:06:50.946690 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 456/900] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:50 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:50.984979 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:50.985675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:50.993359 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:50.996229 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=6 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:51.005086 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:51.005903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:51.006348 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:51.007087 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:51.007713 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:51.008191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:51.205657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:51.214254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:51.216772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:51.218087 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6f5ed475-5dbb-4888-8337-51b9fa8dacd9 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1741 microversion: 2.20 time: 0.237141 Oct 20 09:06:51.218684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 445/901] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:50 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1741 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:51.964181 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4c489385-4f00-4594-8877-c3247335b609 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:51.964815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4c489385-4f00-4594-8877-c3247335b609 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:51.968695 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:51.973329 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c489385-4f00-4594-8877-c3247335b609 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:51.973812 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c489385-4f00-4594-8877-c3247335b609 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:51.974290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c489385-4f00-4594-8877-c3247335b609 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:51.975035 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c489385-4f00-4594-8877-c3247335b609 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:51.975488 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c489385-4f00-4594-8877-c3247335b609 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:51.975916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4c489385-4f00-4594-8877-c3247335b609 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:52.110429 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4c489385-4f00-4594-8877-c3247335b609 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:52.119621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4c489385-4f00-4594-8877-c3247335b609 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.159365 Oct 20 09:06:52.123669 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 457/902] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:51 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:52.243254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:52.243869 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:52.251872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:52.252501 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:52.253323 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:52.254229 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:52.254759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:52.255128 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:52.476776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:52.486949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:52.489850 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:52.491386 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0daf134e-203e-414b-85f7-1c64a07d6f84 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1723 microversion: 2.20 time: 0.252294 Oct 20 09:06:52.493820 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 446/903] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:52 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1723 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:53.143231 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6a952766-f935-45ad-98ee-7cd48a79f990 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:53.143834 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6a952766-f935-45ad-98ee-7cd48a79f990 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:53.150100 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a952766-f935-45ad-98ee-7cd48a79f990 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:53.150609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a952766-f935-45ad-98ee-7cd48a79f990 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:53.151040 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a952766-f935-45ad-98ee-7cd48a79f990 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:53.151719 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a952766-f935-45ad-98ee-7cd48a79f990 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:53.152184 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a952766-f935-45ad-98ee-7cd48a79f990 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:53.152585 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6a952766-f935-45ad-98ee-7cd48a79f990 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:53.289926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6a952766-f935-45ad-98ee-7cd48a79f990 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:53.298961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6a952766-f935-45ad-98ee-7cd48a79f990 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.159440 Oct 20 09:06:53.299539 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 458/904] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:53 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:53.520700 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:53.521341 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:53.527680 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:53.528166 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:53.528656 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:53.529354 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:53.529806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:53.530235 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:53.665056 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:53.674917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:53.676433 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:53.677631 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-de3d4b29-d1d6-4554-8f86-6e16724c93bc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1723 microversion: 2.20 time: 0.160786 Oct 20 09:06:53.678221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 447/905] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:53 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1723 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:54.321122 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-440cf5a4-2d7c-45a6-9dbd-7a7ade11eb4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:54.321766 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-440cf5a4-2d7c-45a6-9dbd-7a7ade11eb4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:54.328248 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-440cf5a4-2d7c-45a6-9dbd-7a7ade11eb4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:54.328729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-440cf5a4-2d7c-45a6-9dbd-7a7ade11eb4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:54.329163 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-440cf5a4-2d7c-45a6-9dbd-7a7ade11eb4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:54.329850 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-440cf5a4-2d7c-45a6-9dbd-7a7ade11eb4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:54.330299 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-440cf5a4-2d7c-45a6-9dbd-7a7ade11eb4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:54.330747 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-440cf5a4-2d7c-45a6-9dbd-7a7ade11eb4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:54.460322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-440cf5a4-2d7c-45a6-9dbd-7a7ade11eb4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:54.468744 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-440cf5a4-2d7c-45a6-9dbd-7a7ade11eb4a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.151487 Oct 20 09:06:54.469310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 459/906] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:54 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:54.704803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:54.705408 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:54.711777 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:54.712255 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:54.712694 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:54.713382 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:54.713834 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:54.714260 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:54.850339 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:54.858622 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:54.860053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:54.861196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-01ab6b8e-32da-4793-bfe8-06866b9a8788 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1723 microversion: 2.20 time: 0.160386 Oct 20 09:06:54.861744 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 448/907] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:54 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1723 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:55.491711 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b5168938-5768-4ef4-a9d9-15c223f14bf2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:55.492315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b5168938-5768-4ef4-a9d9-15c223f14bf2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:55.499172 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5168938-5768-4ef4-a9d9-15c223f14bf2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:55.499655 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5168938-5768-4ef4-a9d9-15c223f14bf2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:55.500087 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5168938-5768-4ef4-a9d9-15c223f14bf2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:55.500776 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5168938-5768-4ef4-a9d9-15c223f14bf2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:55.501225 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5168938-5768-4ef4-a9d9-15c223f14bf2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:55.501649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5168938-5768-4ef4-a9d9-15c223f14bf2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:55.629436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b5168938-5768-4ef4-a9d9-15c223f14bf2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:55.637056 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b5168938-5768-4ef4-a9d9-15c223f14bf2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.149365 Oct 20 09:06:55.637557 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 460/908] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:55 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:55.888822 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:55.889366 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:55.895701 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:55.896177 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:55.896618 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:55.897279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:55.897725 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:55.898140 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.032907 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-95288a9b-4190-434b-bae7-062751c2425e req-5587fa95-d43f-4a20-997a-f8c3f3092836 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7facb789-628a-44aa-b0ec-7f6aee5617dd", "name": "network-vif-unplugged", "status": "completed", "tag": "f17302af-3ee4-46d2-a6fe-a973b183f4e4"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:56.033667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:56.042884 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:56.044501 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:56.045048 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-95288a9b-4190-434b-bae7-062751c2425e req-5587fa95-d43f-4a20-997a-f8c3f3092836 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.045586 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-95288a9b-4190-434b-bae7-062751c2425e req-5587fa95-d43f-4a20-997a-f8c3f3092836 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.045861 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4ab28e56-a16d-4158-b82e-ee27a6693d69 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1666 microversion: 2.20 time: 0.161928 Oct 20 09:06:56.046144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-95288a9b-4190-434b-bae7-062751c2425e req-5587fa95-d43f-4a20-997a-f8c3f3092836 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.046420 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 449/909] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:55 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1666 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:56.074551 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.compute.server_external_events [req-95288a9b-4190-434b-bae7-062751c2425e req-5587fa95-d43f-4a20-997a-f8c3f3092836 service nova] Unable to find a host for instance 7facb789-628a-44aa-b0ec-7f6aee5617dd. Dropping event network-vif-unplugged {{(pid=119067) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 20 09:06:56.074752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f2ffdeaa-94d8-40c2-8bd5-3082375671a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "79a46f8b-389d-412c-876e-5e666c49abb6"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:56.076063 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-95288a9b-4190-434b-bae7-062751c2425e req-5587fa95-d43f-4a20-997a-f8c3f3092836 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.047131 Oct 20 09:06:56.076934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 461/910] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:06:56 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 50 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Oct 20 09:06:56.077672 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f2ffdeaa-94d8-40c2-8bd5-3082375671a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:56.083555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2ffdeaa-94d8-40c2-8bd5-3082375671a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.084037 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2ffdeaa-94d8-40c2-8bd5-3082375671a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.084466 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2ffdeaa-94d8-40c2-8bd5-3082375671a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.085149 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2ffdeaa-94d8-40c2-8bd5-3082375671a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.085593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2ffdeaa-94d8-40c2-8bd5-3082375671a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.086012 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2ffdeaa-94d8-40c2-8bd5-3082375671a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.100710 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:56.376407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f2ffdeaa-94d8-40c2-8bd5-3082375671a8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "POST /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.305511 Oct 20 09:06:56.383097 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 450/911] 10.222.0.36 () {62 vars in 1364 bytes} [Thu Oct 20 09:06:56 2022] POST /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/os-volume_attachments => generated 188 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 20 09:06:56.508199 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-765ec22f-e2b8-4fab-acf9-194d00c218e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:56.508685 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-765ec22f-e2b8-4fab-acf9-194d00c218e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:56.515199 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-765ec22f-e2b8-4fab-acf9-194d00c218e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.515797 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-765ec22f-e2b8-4fab-acf9-194d00c218e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.516367 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-765ec22f-e2b8-4fab-acf9-194d00c218e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.517164 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-765ec22f-e2b8-4fab-acf9-194d00c218e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.517677 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-765ec22f-e2b8-4fab-acf9-194d00c218e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.518157 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-765ec22f-e2b8-4fab-acf9-194d00c218e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.660912 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7a0c9545-7b8b-4ed5-97ea-1c0e4a1694c7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:56.661631 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7a0c9545-7b8b-4ed5-97ea-1c0e4a1694c7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:56.669817 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0c9545-7b8b-4ed5-97ea-1c0e4a1694c7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.670493 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0c9545-7b8b-4ed5-97ea-1c0e4a1694c7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.670940 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0c9545-7b8b-4ed5-97ea-1c0e4a1694c7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.671623 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0c9545-7b8b-4ed5-97ea-1c0e4a1694c7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.672250 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0c9545-7b8b-4ed5-97ea-1c0e4a1694c7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.672673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a0c9545-7b8b-4ed5-97ea-1c0e4a1694c7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.775499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-765ec22f-e2b8-4fab-acf9-194d00c218e8 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "DELETE /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/os-volume_attachments/79a46f8b-389d-412c-876e-5e666c49abb6" status: 202 len: 0 microversion: 2.20 time: 0.271574 Oct 20 09:06:56.777121 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 462/912] 10.222.0.36 () {60 vars in 1457 bytes} [Thu Oct 20 09:06:56 2022] DELETE /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/os-volume_attachments/79a46f8b-389d-412c-876e-5e666c49abb6 => generated 0 bytes in 274 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:06:56.835540 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7a0c9545-7b8b-4ed5-97ea-1c0e4a1694c7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:56.840314 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-73920784-2d15-4415-b2c7-212659c6fe6c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:06:56.841493 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-73920784-2d15-4415-b2c7-212659c6fe6c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:56.844517 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7a0c9545-7b8b-4ed5-97ea-1c0e4a1694c7 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.188225 Oct 20 09:06:56.845189 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 451/913] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:56 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:56.847447 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73920784-2d15-4415-b2c7-212659c6fe6c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.847940 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73920784-2d15-4415-b2c7-212659c6fe6c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.848413 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73920784-2d15-4415-b2c7-212659c6fe6c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.849116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73920784-2d15-4415-b2c7-212659c6fe6c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.849574 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73920784-2d15-4415-b2c7-212659c6fe6c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.849998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73920784-2d15-4415-b2c7-212659c6fe6c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.894267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-73920784-2d15-4415-b2c7-212659c6fe6c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=119067) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 20 09:06:56.962013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-73920784-2d15-4415-b2c7-212659c6fe6c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "POST /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/action" status: 202 len: 0 microversion: 2.20 time: 0.125861 Oct 20 09:06:56.962911 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 463/914] 10.222.0.36 () {62 vars in 1319 bytes} [Thu Oct 20 09:06:56 2022] POST /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/action => generated 0 bytes in 127 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 20 09:06:56.971826 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:56.973162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:56.980587 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.981221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.981923 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:56.982757 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:56.983323 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:56.983848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:57.157479 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:57.166296 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:57.168325 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:57.170957 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1eab0ecc-a126-4ed7-874b-ed83507fac01 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1674 microversion: 2.20 time: 0.203507 Oct 20 09:06:57.171906 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 452/915] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:56 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1674 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:06:57.869833 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bf9b72bf-1db2-4b8c-a270-546274868469 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:57.871143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bf9b72bf-1db2-4b8c-a270-546274868469 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:57.879941 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf9b72bf-1db2-4b8c-a270-546274868469 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:57.880600 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf9b72bf-1db2-4b8c-a270-546274868469 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:57.881087 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf9b72bf-1db2-4b8c-a270-546274868469 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:57.881958 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf9b72bf-1db2-4b8c-a270-546274868469 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:57.882411 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf9b72bf-1db2-4b8c-a270-546274868469 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:57.882912 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bf9b72bf-1db2-4b8c-a270-546274868469 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:58.119697 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bf9b72bf-1db2-4b8c-a270-546274868469 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:58.129201 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bf9b72bf-1db2-4b8c-a270-546274868469 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.263589 Oct 20 09:06:58.129843 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 464/916] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:57 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:58.201896 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:58.202511 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:58.208776 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:58.209483 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:58.210124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:58.210843 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:58.211458 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:58.212151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:58.359692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:58.374569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:58.376031 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:58.377848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-efa57eeb-9ae2-4f38-9082-93535b0399fc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1676 microversion: 2.20 time: 0.180542 Oct 20 09:06:58.385963 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 453/917] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:58 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1676 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:59.151597 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-436f86c5-27ee-4d33-a770-382dde9242b0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:59.152262 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-436f86c5-27ee-4d33-a770-382dde9242b0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:59.159704 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-436f86c5-27ee-4d33-a770-382dde9242b0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:59.160276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-436f86c5-27ee-4d33-a770-382dde9242b0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:59.160762 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-436f86c5-27ee-4d33-a770-382dde9242b0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:59.161519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-436f86c5-27ee-4d33-a770-382dde9242b0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:59.162035 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-436f86c5-27ee-4d33-a770-382dde9242b0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:59.162549 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-436f86c5-27ee-4d33-a770-382dde9242b0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:59.317782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-436f86c5-27ee-4d33-a770-382dde9242b0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:59.325929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-436f86c5-27ee-4d33-a770-382dde9242b0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.177543 Oct 20 09:06:59.326538 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 465/918] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:06:59 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:06:59.413289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:59.414102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:59.420545 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=83,nova_api:INSERT=3 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:59.422903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:59.423406 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:59.424269 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:59.424898 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:59.425419 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:59.425910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:59.634416 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:59.643425 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:59.645109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:06:59.646329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-37decf69-d362-49fb-8e30-4172c287f67e tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.237068 Oct 20 09:06:59.646973 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 454/919] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:06:59 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:06:59.947730 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d2858722-3c14-4f44-99ab-affef91d22df tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:06:59.948462 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d2858722-3c14-4f44-99ab-affef91d22df tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:06:59.956786 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2858722-3c14-4f44-99ab-affef91d22df tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:59.957637 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2858722-3c14-4f44-99ab-affef91d22df tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:59.958728 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2858722-3c14-4f44-99ab-affef91d22df tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:59.961361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2858722-3c14-4f44-99ab-affef91d22df tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:06:59.962906 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2858722-3c14-4f44-99ab-affef91d22df tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:06:59.964401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d2858722-3c14-4f44-99ab-affef91d22df tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:06:59.970397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=3,nova_cell1:UPDATE=5 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:06:59.994667 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d2858722-3c14-4f44-99ab-affef91d22df tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/ips" status: 200 len: 149 microversion: 2.1 time: 0.051105 Oct 20 09:06:59.995964 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 466/920] 10.222.0.36 () {58 vars in 1248 bytes} [Thu Oct 20 09:06:59 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/ips => generated 149 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:00.147562 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-8d5bc5e6-f4a3-4cc1-b50c-d5491e270ffb req-5ec6046f-e84c-4d07-839b-56ab272a6998 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9c52d13f-7238-490c-a0cd-c1d2fff9afb0", "name": "network-vif-plugged", "status": "completed", "tag": "d5aa2d4a-df18-490f-b5c0-9074a308160e"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:00.163648 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-8d5bc5e6-f4a3-4cc1-b50c-d5491e270ffb req-5ec6046f-e84c-4d07-839b-56ab272a6998 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:00.164357 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-8d5bc5e6-f4a3-4cc1-b50c-d5491e270ffb req-5ec6046f-e84c-4d07-839b-56ab272a6998 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:00.164792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-8d5bc5e6-f4a3-4cc1-b50c-d5491e270ffb req-5ec6046f-e84c-4d07-839b-56ab272a6998 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:00.187994 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-8d5bc5e6-f4a3-4cc1-b50c-d5491e270ffb req-5ec6046f-e84c-4d07-839b-56ab272a6998 service nova] Creating event network-vif-plugged:d5aa2d4a-df18-490f-b5c0-9074a308160e for instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 on hv-cinder-maste Oct 20 09:07:00.193677 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-8d5bc5e6-f4a3-4cc1-b50c-d5491e270ffb req-5ec6046f-e84c-4d07-839b-56ab272a6998 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049623 Oct 20 09:07:00.195132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 455/921] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:07:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:00.349687 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-83837f38-35ac-4357-a84e-969fe6cc06a5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:00.351709 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-83837f38-35ac-4357-a84e-969fe6cc06a5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:00.379653 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-83837f38-35ac-4357-a84e-969fe6cc06a5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:00.380173 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-83837f38-35ac-4357-a84e-969fe6cc06a5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:00.380611 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-83837f38-35ac-4357-a84e-969fe6cc06a5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:00.384275 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-83837f38-35ac-4357-a84e-969fe6cc06a5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:00.385358 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-83837f38-35ac-4357-a84e-969fe6cc06a5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:00.387348 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-83837f38-35ac-4357-a84e-969fe6cc06a5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:00.599143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-83837f38-35ac-4357-a84e-969fe6cc06a5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:00.611136 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-83837f38-35ac-4357-a84e-969fe6cc06a5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.265275 Oct 20 09:07:00.612499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 467/922] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:00 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:00.675019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:00.676331 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:00.687294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:00.688050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:00.688821 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:00.689702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:00.690151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:00.690614 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:00.838869 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:00.848582 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:00.850089 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:00.851261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b4657704-9f3f-470d-adc2-5829e069b7b1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.181355 Oct 20 09:07:00.851826 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 456/923] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:00 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:01.494361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-21701193-364c-4cbc-9ec4-49dec28fac97 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "da84f2e4-b9e8-480d-a560-ab4f6aa0ecd9"}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:01.498364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-21701193-364c-4cbc-9ec4-49dec28fac97 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:01.505237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21701193-364c-4cbc-9ec4-49dec28fac97 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:01.506061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21701193-364c-4cbc-9ec4-49dec28fac97 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:01.506743 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21701193-364c-4cbc-9ec4-49dec28fac97 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:01.507974 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21701193-364c-4cbc-9ec4-49dec28fac97 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:01.508648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21701193-364c-4cbc-9ec4-49dec28fac97 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:01.509398 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-21701193-364c-4cbc-9ec4-49dec28fac97 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:01.638088 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-76d58ae0-fe1a-4641-8eaf-210ce74b41c1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:01.638854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-76d58ae0-fe1a-4641-8eaf-210ce74b41c1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:01.650227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76d58ae0-fe1a-4641-8eaf-210ce74b41c1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:01.650227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76d58ae0-fe1a-4641-8eaf-210ce74b41c1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:01.650227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76d58ae0-fe1a-4641-8eaf-210ce74b41c1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:01.650227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76d58ae0-fe1a-4641-8eaf-210ce74b41c1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:01.650583 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76d58ae0-fe1a-4641-8eaf-210ce74b41c1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:01.650583 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-76d58ae0-fe1a-4641-8eaf-210ce74b41c1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:01.819593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-76d58ae0-fe1a-4641-8eaf-210ce74b41c1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:01.834050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-76d58ae0-fe1a-4641-8eaf-210ce74b41c1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.200578 Oct 20 09:07:01.835021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 457/924] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:01 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:07:01.877047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:01.877781 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:01.884499 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:01.885153 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:01.885703 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:01.886564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:01.887112 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:01.888333 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:02.033901 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:02.042943 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:02.045102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:02.046276 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c5650c4a-5ed9-4989-8805-4b4fbf91848c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.173169 Oct 20 09:07:02.046872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 458/925] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:01 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:02.858004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-56c0c54b-3c75-43ba-babe-f6c11ed4f566 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:02.858772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-56c0c54b-3c75-43ba-babe-f6c11ed4f566 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:02.865444 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c0c54b-3c75-43ba-babe-f6c11ed4f566 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:02.865924 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c0c54b-3c75-43ba-babe-f6c11ed4f566 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:02.866354 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c0c54b-3c75-43ba-babe-f6c11ed4f566 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:02.867069 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c0c54b-3c75-43ba-babe-f6c11ed4f566 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:02.867512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c0c54b-3c75-43ba-babe-f6c11ed4f566 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:02.867929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56c0c54b-3c75-43ba-babe-f6c11ed4f566 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:03.005135 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-56c0c54b-3c75-43ba-babe-f6c11ed4f566 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:03.013987 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-56c0c54b-3c75-43ba-babe-f6c11ed4f566 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.159790 Oct 20 09:07:03.030768 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 459/926] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:02 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:07:03.072192 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:03.072806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:03.079254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:03.079918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:03.080278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:03.081151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:03.081772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:03.082232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:03.248170 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:03.263841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:03.265529 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:03.266926 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-45bc73cd-0092-4d84-a063-40a6c4a437f0 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.199143 Oct 20 09:07:03.267492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 460/927] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:03 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:07:04.037322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-08d5f8bb-95bb-416b-9c55-c7bcb1a9f5eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:04.038966 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-08d5f8bb-95bb-416b-9c55-c7bcb1a9f5eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:04.046363 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-08d5f8bb-95bb-416b-9c55-c7bcb1a9f5eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:04.046896 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-08d5f8bb-95bb-416b-9c55-c7bcb1a9f5eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:04.047381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-08d5f8bb-95bb-416b-9c55-c7bcb1a9f5eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:04.048094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-08d5f8bb-95bb-416b-9c55-c7bcb1a9f5eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:04.048892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-08d5f8bb-95bb-416b-9c55-c7bcb1a9f5eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:04.049543 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-08d5f8bb-95bb-416b-9c55-c7bcb1a9f5eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:04.237897 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-08d5f8bb-95bb-416b-9c55-c7bcb1a9f5eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:04.245648 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-08d5f8bb-95bb-416b-9c55-c7bcb1a9f5eb tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.212379 Oct 20 09:07:04.246224 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 461/928] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:04 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:07:04.301563 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:04.302195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:04.312371 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:04.312879 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:04.313317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:04.314172 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:04.314665 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:04.315089 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:04.489443 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:04.498442 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:04.499921 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:04.501092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c65973df-4438-4ee4-acf4-83b730450370 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.208865 Oct 20 09:07:04.501658 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 462/929] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:04 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:05.271453 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-32ad934c-1c53-4ce5-84ec-ab130dfe58aa tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:05.273055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-32ad934c-1c53-4ce5-84ec-ab130dfe58aa tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:05.280868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32ad934c-1c53-4ce5-84ec-ab130dfe58aa tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:05.281362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32ad934c-1c53-4ce5-84ec-ab130dfe58aa tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:05.281805 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32ad934c-1c53-4ce5-84ec-ab130dfe58aa tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:05.282527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32ad934c-1c53-4ce5-84ec-ab130dfe58aa tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:05.283012 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32ad934c-1c53-4ce5-84ec-ab130dfe58aa tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:05.283438 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-32ad934c-1c53-4ce5-84ec-ab130dfe58aa tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:05.530631 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-32ad934c-1c53-4ce5-84ec-ab130dfe58aa tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:05.539486 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-32ad934c-1c53-4ce5-84ec-ab130dfe58aa tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.273753 Oct 20 09:07:05.540066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 463/930] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:05 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:05.545290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:05.545883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:05.552109 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:05.552803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:05.553406 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:05.554282 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:05.554918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:05.555507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:05.729769 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:05.739471 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:05.740916 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:05.742233 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8f0ba3c1-f52c-4a5d-a395-8551b7b3b77b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.201077 Oct 20 09:07:05.743290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 464/931] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:05 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:06.563529 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c008671b-1f3a-4e71-a119-3e2232074ad2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:06.565017 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c008671b-1f3a-4e71-a119-3e2232074ad2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:06.573252 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c008671b-1f3a-4e71-a119-3e2232074ad2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:06.573794 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c008671b-1f3a-4e71-a119-3e2232074ad2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:06.574233 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c008671b-1f3a-4e71-a119-3e2232074ad2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:06.574996 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c008671b-1f3a-4e71-a119-3e2232074ad2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:06.575453 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c008671b-1f3a-4e71-a119-3e2232074ad2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:06.576047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c008671b-1f3a-4e71-a119-3e2232074ad2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:06.580553 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:07:06.818083 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c008671b-1f3a-4e71-a119-3e2232074ad2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:06.827550 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c008671b-1f3a-4e71-a119-3e2232074ad2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.268395 Oct 20 09:07:06.828132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 465/932] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:06 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:06.839349 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:06.840291 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:06.847281 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:06.847769 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:06.848195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:06.848885 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:06.849329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:06.849745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:06.989019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:06.997919 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:06.999383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:07.000667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a08c4ed4-4a11-44df-89ca-e58e50867026 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.165031 Oct 20 09:07:07.001438 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 466/933] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:06 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:07.852964 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7e0b29be-c1ee-4eeb-9438-234cf8a16a49 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:07.853590 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7e0b29be-c1ee-4eeb-9438-234cf8a16a49 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:07.861673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7e0b29be-c1ee-4eeb-9438-234cf8a16a49 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:07.862425 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7e0b29be-c1ee-4eeb-9438-234cf8a16a49 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:07.863148 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7e0b29be-c1ee-4eeb-9438-234cf8a16a49 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:07.864249 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7e0b29be-c1ee-4eeb-9438-234cf8a16a49 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:07.864991 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7e0b29be-c1ee-4eeb-9438-234cf8a16a49 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:07.865650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7e0b29be-c1ee-4eeb-9438-234cf8a16a49 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:08.035372 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7e0b29be-c1ee-4eeb-9438-234cf8a16a49 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:08.044650 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7e0b29be-c1ee-4eeb-9438-234cf8a16a49 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.195431 Oct 20 09:07:08.045492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 467/934] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:07 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:07:08.050343 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:08.050899 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:08.057007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:08.057713 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:08.058328 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:08.059745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:08.060345 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:08.060920 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:08.205937 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:08.217105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:08.218521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:08.219652 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a7b72783-f4ee-464b-a10e-ac16b9e183ff tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.173109 Oct 20 09:07:08.220205 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 468/935] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:08 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:09.068326 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0beaa3cb-f744-4785-9ab2-265fe2b8d459 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:09.068928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-0beaa3cb-f744-4785-9ab2-265fe2b8d459 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:09.076254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0beaa3cb-f744-4785-9ab2-265fe2b8d459 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:09.076747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0beaa3cb-f744-4785-9ab2-265fe2b8d459 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:09.077192 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0beaa3cb-f744-4785-9ab2-265fe2b8d459 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:09.083122 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0beaa3cb-f744-4785-9ab2-265fe2b8d459 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:09.083597 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0beaa3cb-f744-4785-9ab2-265fe2b8d459 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:09.084027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-0beaa3cb-f744-4785-9ab2-265fe2b8d459 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:09.242638 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-0beaa3cb-f744-4785-9ab2-265fe2b8d459 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:09.252951 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0beaa3cb-f744-4785-9ab2-265fe2b8d459 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.188263 Oct 20 09:07:09.268545 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 469/936] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:09 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:07:09.276300 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:09.276904 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:09.283738 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:09.284233 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:09.284666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:09.285367 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:09.286003 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:09.286430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:09.439368 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:09.448416 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:09.450481 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:09.451659 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b9f5648c-c34b-4db5-811d-9566b3286ef4 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.182076 Oct 20 09:07:09.452220 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 470/937] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:09 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:10.279140 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ffc9a83e-0fa3-4f60-9a80-3c1781b89e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:10.279747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ffc9a83e-0fa3-4f60-9a80-3c1781b89e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:10.286111 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffc9a83e-0fa3-4f60-9a80-3c1781b89e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:10.286632 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffc9a83e-0fa3-4f60-9a80-3c1781b89e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:10.287094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffc9a83e-0fa3-4f60-9a80-3c1781b89e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:10.287752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffc9a83e-0fa3-4f60-9a80-3c1781b89e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:10.288207 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffc9a83e-0fa3-4f60-9a80-3c1781b89e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:10.288629 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ffc9a83e-0fa3-4f60-9a80-3c1781b89e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:10.430004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ffc9a83e-0fa3-4f60-9a80-3c1781b89e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:10.438191 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ffc9a83e-0fa3-4f60-9a80-3c1781b89e8e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.163537 Oct 20 09:07:10.438978 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 471/938] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:10 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:07:10.477768 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:10.478498 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:10.484625 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:10.485101 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:10.485527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:10.486203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:10.486678 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:10.487095 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:10.621454 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:10.630414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:10.632101 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:10.633408 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b9747fa9-5923-4f09-a3e3-2e69f7b9a581 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.159690 Oct 20 09:07:10.634034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 472/939] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:10 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:07:11.213028 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-21701193-364c-4cbc-9ec4-49dec28fac97 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "POST /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 9.722523 Oct 20 09:07:11.213712 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 468/940] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:07:01 2022] POST /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments => generated 194 bytes in 9724 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:11.461504 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-472c5ba5-2cf3-458e-ba1e-84b9916a86dc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:11.462116 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-472c5ba5-2cf3-458e-ba1e-84b9916a86dc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:11.468597 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-472c5ba5-2cf3-458e-ba1e-84b9916a86dc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:11.469089 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-472c5ba5-2cf3-458e-ba1e-84b9916a86dc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:11.469527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-472c5ba5-2cf3-458e-ba1e-84b9916a86dc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:11.470684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-472c5ba5-2cf3-458e-ba1e-84b9916a86dc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:11.471570 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-472c5ba5-2cf3-458e-ba1e-84b9916a86dc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:11.472034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-472c5ba5-2cf3-458e-ba1e-84b9916a86dc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:11.608551 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-472c5ba5-2cf3-458e-ba1e-84b9916a86dc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:11.616279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-472c5ba5-2cf3-458e-ba1e-84b9916a86dc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1266 microversion: 2.1 time: 0.158532 Oct 20 09:07:11.616848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 473/941] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:11 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1266 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:11.657277 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=57,nova_api:UPDATE=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:07:11.661530 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:11.662307 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:11.670125 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:11.670653 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:11.671098 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:11.671801 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:11.672255 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:11.672698 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:11.804262 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:11.813568 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:11.815167 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:11.816420 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-433cdc9d-cf1b-4f1b-9bc4-45eead250b05 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.160497 Oct 20 09:07:11.817217 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 469/942] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:11 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:12.638796 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e8858bba-2791-41d0-9dfa-7fa69845bfa5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:12.639405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e8858bba-2791-41d0-9dfa-7fa69845bfa5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:12.646023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8858bba-2791-41d0-9dfa-7fa69845bfa5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:12.646580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8858bba-2791-41d0-9dfa-7fa69845bfa5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:12.647091 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8858bba-2791-41d0-9dfa-7fa69845bfa5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:12.647847 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8858bba-2791-41d0-9dfa-7fa69845bfa5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:12.648302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8858bba-2791-41d0-9dfa-7fa69845bfa5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:12.648735 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8858bba-2791-41d0-9dfa-7fa69845bfa5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:12.828012 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e8858bba-2791-41d0-9dfa-7fa69845bfa5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:12.838331 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e8858bba-2791-41d0-9dfa-7fa69845bfa5 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1453 microversion: 2.1 time: 0.203237 Oct 20 09:07:12.838956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 474/943] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:12 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1453 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:12.844538 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:12.845822 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:12.853107 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:12.853908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:12.854549 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:12.855246 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:12.855729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:12.857009 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:12.867621 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d321f297-57ac-48b4-b7d5-eda7f209b624 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:12.871649 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d321f297-57ac-48b4-b7d5-eda7f209b624 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:12.878694 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d321f297-57ac-48b4-b7d5-eda7f209b624 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:12.879561 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d321f297-57ac-48b4-b7d5-eda7f209b624 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:12.880043 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d321f297-57ac-48b4-b7d5-eda7f209b624 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:12.881125 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d321f297-57ac-48b4-b7d5-eda7f209b624 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:12.881659 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d321f297-57ac-48b4-b7d5-eda7f209b624 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:12.882268 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d321f297-57ac-48b4-b7d5-eda7f209b624 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:13.008768 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:13.018061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:13.019826 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:13.021082 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0d2fcf6f-aa2c-4ab0-b561-a6fe79b63233 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.181889 Oct 20 09:07:13.021853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 470/944] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:12 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:13.050573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-d321f297-57ac-48b4-b7d5-eda7f209b624 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:13.058448 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d321f297-57ac-48b4-b7d5-eda7f209b624 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1453 microversion: 2.1 time: 0.199487 Oct 20 09:07:13.060601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 475/945] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:12 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1453 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:13.474990 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4e0067b9-3295-41a3-aa5e-ef75cee0003d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:13.475718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4e0067b9-3295-41a3-aa5e-ef75cee0003d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:13.481981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e0067b9-3295-41a3-aa5e-ef75cee0003d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:13.482490 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e0067b9-3295-41a3-aa5e-ef75cee0003d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:13.482979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e0067b9-3295-41a3-aa5e-ef75cee0003d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:13.483735 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e0067b9-3295-41a3-aa5e-ef75cee0003d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:13.484816 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e0067b9-3295-41a3-aa5e-ef75cee0003d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:13.485252 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4e0067b9-3295-41a3-aa5e-ef75cee0003d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:13.510627 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4e0067b9-3295-41a3-aa5e-ef75cee0003d tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/ips" status: 200 len: 146 microversion: 2.1 time: 0.326560 Oct 20 09:07:13.514582 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 471/946] 10.222.0.36 () {58 vars in 1248 bytes} [Thu Oct 20 09:07:13 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/ips => generated 146 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:14.051227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:14.053201 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:14.064195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:14.065751 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:14.066202 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:14.067104 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:14.067729 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:14.068446 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:14.245982 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:14.257777 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:14.259995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:14.261775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f50c14a8-c260-47eb-a826-36f1dcea0b19 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.215500 Oct 20 09:07:14.263309 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 476/947] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:14 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:14.873658 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-99425220-7228-40aa-83e5-d5618d2fdf9f req-d80eb7e2-cfed-46f5-a93e-f920ea7c5bfc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9c52d13f-7238-490c-a0cd-c1d2fff9afb0", "tag": "d5aa2d4a-df18-490f-b5c0-9074a308160e"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:14.889610 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-99425220-7228-40aa-83e5-d5618d2fdf9f req-d80eb7e2-cfed-46f5-a93e-f920ea7c5bfc service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:14.890620 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-99425220-7228-40aa-83e5-d5618d2fdf9f req-d80eb7e2-cfed-46f5-a93e-f920ea7c5bfc service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:14.891439 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-99425220-7228-40aa-83e5-d5618d2fdf9f req-d80eb7e2-cfed-46f5-a93e-f920ea7c5bfc service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:14.910986 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-99425220-7228-40aa-83e5-d5618d2fdf9f req-d80eb7e2-cfed-46f5-a93e-f920ea7c5bfc service nova] Creating event network-changed:d5aa2d4a-df18-490f-b5c0-9074a308160e for instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 on hv-cinder-maste Oct 20 09:07:14.916783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-99425220-7228-40aa-83e5-d5618d2fdf9f req-d80eb7e2-cfed-46f5-a93e-f920ea7c5bfc service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047338 Oct 20 09:07:14.917618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 472/948] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:07:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:14.920332 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-da276595-b2e2-4f11-821f-edf23f8496e2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b1034f78-9144-40e6-aa02-158b3e3d51e5"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:14.923194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-da276595-b2e2-4f11-821f-edf23f8496e2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:14.929442 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da276595-b2e2-4f11-821f-edf23f8496e2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:14.929921 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da276595-b2e2-4f11-821f-edf23f8496e2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:14.930353 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da276595-b2e2-4f11-821f-edf23f8496e2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:14.931066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da276595-b2e2-4f11-821f-edf23f8496e2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:14.931507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da276595-b2e2-4f11-821f-edf23f8496e2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:14.931927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da276595-b2e2-4f11-821f-edf23f8496e2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:15.235589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-da276595-b2e2-4f11-821f-edf23f8496e2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.320921 Oct 20 09:07:15.237111 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 477/949] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:07:14 2022] POST /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments => generated 194 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:15.290305 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:15.290990 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:15.303450 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:15.303944 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:15.304381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:15.305087 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:15.305540 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:15.305966 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:15.468928 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:15.477764 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:15.479236 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:15.480398 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6d3b2e20-7c24-4252-b62c-e91319a439b7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.194879 Oct 20 09:07:15.481592 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 473/950] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:15 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:16.202188 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-bc09589e-7fdd-40c9-8fb8-4eec3144c205 req-a957d621-852c-4b78-afba-6a9670ed8857 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7facb789-628a-44aa-b0ec-7f6aee5617dd", "name": "network-vif-plugged", "status": "completed", "tag": "f17302af-3ee4-46d2-a6fe-a973b183f4e4"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:16.213687 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-bc09589e-7fdd-40c9-8fb8-4eec3144c205 req-a957d621-852c-4b78-afba-6a9670ed8857 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:16.214173 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-bc09589e-7fdd-40c9-8fb8-4eec3144c205 req-a957d621-852c-4b78-afba-6a9670ed8857 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:16.214638 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-bc09589e-7fdd-40c9-8fb8-4eec3144c205 req-a957d621-852c-4b78-afba-6a9670ed8857 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:16.226068 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-bc09589e-7fdd-40c9-8fb8-4eec3144c205 req-a957d621-852c-4b78-afba-6a9670ed8857 service nova] Creating event network-vif-plugged:f17302af-3ee4-46d2-a6fe-a973b183f4e4 for instance 7facb789-628a-44aa-b0ec-7f6aee5617dd on hv-cinder-maste Oct 20 09:07:16.231788 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-bc09589e-7fdd-40c9-8fb8-4eec3144c205 req-a957d621-852c-4b78-afba-6a9670ed8857 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033407 Oct 20 09:07:16.232387 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 478/951] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:07:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:16.506533 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:16.507135 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:16.514031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:16.514529 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:16.514962 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:16.515636 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:16.516079 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:16.516499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:16.641442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:16.649489 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:16.650954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:16.652089 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-fead8f05-2650-4d84-ac4f-ac89cee3fc17 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.149452 Oct 20 09:07:16.652644 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 474/952] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:16 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:17.677576 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:17.678293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:17.685376 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:17.685850 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:17.686276 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:17.686976 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:17.687421 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:17.687840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:17.815276 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:17.823290 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:17.825368 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:17.826542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-cbceb3f5-f60d-48dc-b255-bfcb83e799e5 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.153173 Oct 20 09:07:17.827099 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 479/953] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:17 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:18.851666 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:18.852266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:18.859378 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:18.859850 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:18.860280 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:18.860955 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:18.861398 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:18.861813 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:18.990045 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:18.998154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:18.999621 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:19.000764 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-80c76da8-6a1e-4a8b-8363-9b4f74f09298 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.153038 Oct 20 09:07:19.001324 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 475/954] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:18 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:20.026380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:20.026972 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:20.034414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:20.034927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:20.035392 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:20.036070 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:20.036515 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:20.036961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:20.165870 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:20.173820 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:20.175256 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:20.176397 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7ee985f4-fe77-48e2-be9b-1e4b7f5a85cc tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.153982 Oct 20 09:07:20.176953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 480/955] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:20 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:21.098037 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-befce10f-f848-4f45-a934-8ee8cec05e0c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:21.098643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-befce10f-f848-4f45-a934-8ee8cec05e0c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:21.105020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-befce10f-f848-4f45-a934-8ee8cec05e0c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:21.106123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-befce10f-f848-4f45-a934-8ee8cec05e0c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:21.106588 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-befce10f-f848-4f45-a934-8ee8cec05e0c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:21.107274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-befce10f-f848-4f45-a934-8ee8cec05e0c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:21.107720 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-befce10f-f848-4f45-a934-8ee8cec05e0c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:21.108138 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-befce10f-f848-4f45-a934-8ee8cec05e0c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:21.190033 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-befce10f-f848-4f45-a934-8ee8cec05e0c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lazy-loading 'services' on Instance uuid c5dba102-6a02-40f4-a502-d1fc1cdab301 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:07:21.202067 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:21.202767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:21.210267 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:21.211000 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:21.211440 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:21.212138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:21.212640 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:21.213426 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:21.218780 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:07:21.267929 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-befce10f-f848-4f45-a934-8ee8cec05e0c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments/da84f2e4-b9e8-480d-a560-ab4f6aa0ecd9" status: 202 len: 0 microversion: 2.1 time: 0.173751 Oct 20 09:07:21.268545 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 476/956] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:07:21 2022] DELETE /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments/da84f2e4-b9e8-480d-a560-ab4f6aa0ecd9 => generated 0 bytes in 175 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:07:21.278272 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-643cc4d0-ad86-4751-a546-a0ca396e7115 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:21.280778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-643cc4d0-ad86-4751-a546-a0ca396e7115 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:21.287780 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-643cc4d0-ad86-4751-a546-a0ca396e7115 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:21.288295 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-643cc4d0-ad86-4751-a546-a0ca396e7115 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:21.288713 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-643cc4d0-ad86-4751-a546-a0ca396e7115 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:21.289427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-643cc4d0-ad86-4751-a546-a0ca396e7115 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:21.289881 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-643cc4d0-ad86-4751-a546-a0ca396e7115 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:21.290308 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-643cc4d0-ad86-4751-a546-a0ca396e7115 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:21.326313 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-643cc4d0-ad86-4751-a546-a0ca396e7115 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052960 Oct 20 09:07:21.327135 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 477/957] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:21 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:07:21.496008 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:21.504692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:21.506140 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:21.507325 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3e2d82f7-a21d-4350-b082-76678e35d5ce tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1732 microversion: 2.20 time: 0.309694 Oct 20 09:07:21.507890 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 481/958] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:21 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1732 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:07:22.339316 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8986c518-af7c-4778-aa94-7c6b8f874369 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:22.342046 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8986c518-af7c-4778-aa94-7c6b8f874369 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:22.348794 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8986c518-af7c-4778-aa94-7c6b8f874369 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:22.349274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8986c518-af7c-4778-aa94-7c6b8f874369 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:22.349703 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8986c518-af7c-4778-aa94-7c6b8f874369 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:22.350386 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8986c518-af7c-4778-aa94-7c6b8f874369 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:22.350875 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8986c518-af7c-4778-aa94-7c6b8f874369 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:22.351289 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8986c518-af7c-4778-aa94-7c6b8f874369 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:22.382904 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8986c518-af7c-4778-aa94-7c6b8f874369 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046868 Oct 20 09:07:22.383317 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 478/959] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:22 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:22.533465 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:22.534085 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:22.540651 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:22.541138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:22.541571 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:22.542265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:22.542783 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:22.543792 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:22.678329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:22.687115 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:22.688574 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:22.689771 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-22bfe209-43ac-4094-9c1a-ef8048cc8f4b tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1719 microversion: 2.20 time: 0.160205 Oct 20 09:07:22.690524 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 482/960] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:07:22 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1719 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:07:23.395605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-36118c2c-48e1-4bd5-8d6d-4af9755bc02f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:23.398071 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-36118c2c-48e1-4bd5-8d6d-4af9755bc02f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:23.405680 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-36118c2c-48e1-4bd5-8d6d-4af9755bc02f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:23.406241 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-36118c2c-48e1-4bd5-8d6d-4af9755bc02f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:23.406709 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-36118c2c-48e1-4bd5-8d6d-4af9755bc02f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:23.407489 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-36118c2c-48e1-4bd5-8d6d-4af9755bc02f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:23.408047 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-36118c2c-48e1-4bd5-8d6d-4af9755bc02f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:23.408613 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-36118c2c-48e1-4bd5-8d6d-4af9755bc02f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:23.441024 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-36118c2c-48e1-4bd5-8d6d-4af9755bc02f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049527 Oct 20 09:07:23.441940 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 479/961] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:23 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:24.454644 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7a15b21c-d51f-45e1-8fca-6e927fd87b59 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:24.457523 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7a15b21c-d51f-45e1-8fca-6e927fd87b59 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:24.464637 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a15b21c-d51f-45e1-8fca-6e927fd87b59 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:24.465755 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a15b21c-d51f-45e1-8fca-6e927fd87b59 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:24.466408 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a15b21c-d51f-45e1-8fca-6e927fd87b59 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:24.467212 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a15b21c-d51f-45e1-8fca-6e927fd87b59 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:24.467662 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a15b21c-d51f-45e1-8fca-6e927fd87b59 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:24.468080 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7a15b21c-d51f-45e1-8fca-6e927fd87b59 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:24.501770 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7a15b21c-d51f-45e1-8fca-6e927fd87b59 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051269 Oct 20 09:07:24.502720 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 483/962] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:24 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:25.515302 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7ca6aba0-cff2-4116-b272-e3c010722a8f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:25.518109 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7ca6aba0-cff2-4116-b272-e3c010722a8f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:25.524809 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7ca6aba0-cff2-4116-b272-e3c010722a8f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:25.525442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7ca6aba0-cff2-4116-b272-e3c010722a8f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:25.526072 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7ca6aba0-cff2-4116-b272-e3c010722a8f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:25.526965 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7ca6aba0-cff2-4116-b272-e3c010722a8f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:25.528094 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7ca6aba0-cff2-4116-b272-e3c010722a8f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:25.528677 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7ca6aba0-cff2-4116-b272-e3c010722a8f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:25.559324 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7ca6aba0-cff2-4116-b272-e3c010722a8f tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048513 Oct 20 09:07:25.559894 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 480/963] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:25 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:26.571843 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b8855d74-9d68-4834-b9d8-707a93612cd0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:26.575249 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b8855d74-9d68-4834-b9d8-707a93612cd0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:26.582520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8855d74-9d68-4834-b9d8-707a93612cd0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:26.583011 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8855d74-9d68-4834-b9d8-707a93612cd0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:26.583623 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8855d74-9d68-4834-b9d8-707a93612cd0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:26.584324 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8855d74-9d68-4834-b9d8-707a93612cd0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:26.584766 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8855d74-9d68-4834-b9d8-707a93612cd0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:26.592196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b8855d74-9d68-4834-b9d8-707a93612cd0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:26.627416 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b8855d74-9d68-4834-b9d8-707a93612cd0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059527 Oct 20 09:07:26.628196 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 484/964] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:26 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:27.091414 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f55ac4a8-fe32-40e9-8fe4-5199c3be90b7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:27.093762 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f55ac4a8-fe32-40e9-8fe4-5199c3be90b7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:27.100582 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f55ac4a8-fe32-40e9-8fe4-5199c3be90b7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:27.101166 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f55ac4a8-fe32-40e9-8fe4-5199c3be90b7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:27.101826 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f55ac4a8-fe32-40e9-8fe4-5199c3be90b7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:27.102636 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f55ac4a8-fe32-40e9-8fe4-5199c3be90b7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:27.103162 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f55ac4a8-fe32-40e9-8fe4-5199c3be90b7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:27.103664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f55ac4a8-fe32-40e9-8fe4-5199c3be90b7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:27.148153 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f55ac4a8-fe32-40e9-8fe4-5199c3be90b7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061475 Oct 20 09:07:27.148736 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 481/965] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:27 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:07:27.160126 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d6307561-bbb9-45a5-a93f-b680cc70ed0a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:27.161420 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d6307561-bbb9-45a5-a93f-b680cc70ed0a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:27.169810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d6307561-bbb9-45a5-a93f-b680cc70ed0a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:27.170678 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d6307561-bbb9-45a5-a93f-b680cc70ed0a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:27.171348 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d6307561-bbb9-45a5-a93f-b680cc70ed0a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:27.172144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d6307561-bbb9-45a5-a93f-b680cc70ed0a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:27.172674 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d6307561-bbb9-45a5-a93f-b680cc70ed0a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:27.173328 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d6307561-bbb9-45a5-a93f-b680cc70ed0a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:27.206642 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d6307561-bbb9-45a5-a93f-b680cc70ed0a tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments/b1034f78-9144-40e6-aa02-158b3e3d51e5" status: 200 len: 194 microversion: 2.1 time: 0.050733 Oct 20 09:07:27.207654 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 485/966] 10.222.0.36 () {58 vars in 1413 bytes} [Thu Oct 20 09:07:27 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments/b1034f78-9144-40e6-aa02-158b3e3d51e5 => generated 194 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:27.640749 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c75cfd20-e30d-4952-913b-c48cf5beec43 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:27.643218 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c75cfd20-e30d-4952-913b-c48cf5beec43 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:27.653988 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c75cfd20-e30d-4952-913b-c48cf5beec43 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:27.655975 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c75cfd20-e30d-4952-913b-c48cf5beec43 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:27.656642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c75cfd20-e30d-4952-913b-c48cf5beec43 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:27.658912 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c75cfd20-e30d-4952-913b-c48cf5beec43 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:27.659712 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c75cfd20-e30d-4952-913b-c48cf5beec43 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:27.660269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c75cfd20-e30d-4952-913b-c48cf5beec43 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:27.664563 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:07:27.695817 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c75cfd20-e30d-4952-913b-c48cf5beec43 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059828 Oct 20 09:07:27.696466 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 482/967] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:27 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:28.693694 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f8cb38f0-775c-4ec1-ba68-5d5890552f6f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f409e45c-c769-41e7-bc65-2b8aefe5d85e"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:28.696895 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f8cb38f0-775c-4ec1-ba68-5d5890552f6f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:28.706309 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8cb38f0-775c-4ec1-ba68-5d5890552f6f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:28.706975 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8cb38f0-775c-4ec1-ba68-5d5890552f6f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:28.707576 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8cb38f0-775c-4ec1-ba68-5d5890552f6f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:28.708436 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8cb38f0-775c-4ec1-ba68-5d5890552f6f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:28.708617 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c1ccac75-4270-4e24-93e0-3afa673daf26 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:28.709519 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8cb38f0-775c-4ec1-ba68-5d5890552f6f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:28.710210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f8cb38f0-775c-4ec1-ba68-5d5890552f6f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:28.711155 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c1ccac75-4270-4e24-93e0-3afa673daf26 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:28.718892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c1ccac75-4270-4e24-93e0-3afa673daf26 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:28.719380 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c1ccac75-4270-4e24-93e0-3afa673daf26 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:28.719807 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c1ccac75-4270-4e24-93e0-3afa673daf26 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:28.720534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c1ccac75-4270-4e24-93e0-3afa673daf26 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:28.720983 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c1ccac75-4270-4e24-93e0-3afa673daf26 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:28.721409 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c1ccac75-4270-4e24-93e0-3afa673daf26 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:28.764569 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c1ccac75-4270-4e24-93e0-3afa673daf26 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.059858 Oct 20 09:07:28.770945 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 483/968] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:28 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301/os-volume_attachments => generated 25 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 20 09:07:28.868805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9e20cfe1-5898-4c07-8ba7-9b4b39809fcb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:28.869592 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9e20cfe1-5898-4c07-8ba7-9b4b39809fcb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:28.877099 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9e20cfe1-5898-4c07-8ba7-9b4b39809fcb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:28.877982 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9e20cfe1-5898-4c07-8ba7-9b4b39809fcb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:28.878768 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9e20cfe1-5898-4c07-8ba7-9b4b39809fcb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:28.879664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9e20cfe1-5898-4c07-8ba7-9b4b39809fcb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:28.880277 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9e20cfe1-5898-4c07-8ba7-9b4b39809fcb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:28.880858 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9e20cfe1-5898-4c07-8ba7-9b4b39809fcb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:28.912968 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9e20cfe1-5898-4c07-8ba7-9b4b39809fcb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:07:29.025325 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9e20cfe1-5898-4c07-8ba7-9b4b39809fcb tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 204 len: 0 microversion: 2.1 time: 0.157756 Oct 20 09:07:29.027025 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 484/969] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:07:28 2022] DELETE /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:07:29.036581 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e2bd76e3-fe47-4582-8cfd-c86bb1610afe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:29.039450 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-e2bd76e3-fe47-4582-8cfd-c86bb1610afe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:29.050903 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e2bd76e3-fe47-4582-8cfd-c86bb1610afe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:29.052920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e2bd76e3-fe47-4582-8cfd-c86bb1610afe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:29.054353 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e2bd76e3-fe47-4582-8cfd-c86bb1610afe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:29.057098 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e2bd76e3-fe47-4582-8cfd-c86bb1610afe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:29.057992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e2bd76e3-fe47-4582-8cfd-c86bb1610afe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:29.059637 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e2bd76e3-fe47-4582-8cfd-c86bb1610afe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:29.164063 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f8cb38f0-775c-4ec1-ba68-5d5890552f6f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "POST /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.474061 Oct 20 09:07:29.168742 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 486/970] 10.222.0.36 () {60 vars in 1323 bytes} [Thu Oct 20 09:07:28 2022] POST /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments => generated 194 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:29.392353 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e2bd76e3-fe47-4582-8cfd-c86bb1610afe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:29.407753 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e2bd76e3-fe47-4582-8cfd-c86bb1610afe tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1678 microversion: 2.1 time: 0.375744 Oct 20 09:07:29.408586 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 485/971] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:29 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1678 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:30.433024 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-77f3e200-ba8c-4e06-828d-3c5e90fd278b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:30.434359 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-77f3e200-ba8c-4e06-828d-3c5e90fd278b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:30.436407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=66 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:07:30.441582 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77f3e200-ba8c-4e06-828d-3c5e90fd278b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:30.442072 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77f3e200-ba8c-4e06-828d-3c5e90fd278b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:30.442710 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77f3e200-ba8c-4e06-828d-3c5e90fd278b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:30.443422 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77f3e200-ba8c-4e06-828d-3c5e90fd278b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:30.443877 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77f3e200-ba8c-4e06-828d-3c5e90fd278b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:30.444302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-77f3e200-ba8c-4e06-828d-3c5e90fd278b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:30.620390 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-77f3e200-ba8c-4e06-828d-3c5e90fd278b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:30.631084 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-77f3e200-ba8c-4e06-828d-3c5e90fd278b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1678 microversion: 2.1 time: 0.202872 Oct 20 09:07:30.649981 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 487/972] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:30 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1678 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:31.655697 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8513446f-f004-4614-ba3e-878dacd4b823 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:31.656365 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8513446f-f004-4614-ba3e-878dacd4b823 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:31.666434 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8513446f-f004-4614-ba3e-878dacd4b823 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:31.667230 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8513446f-f004-4614-ba3e-878dacd4b823 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:31.668448 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8513446f-f004-4614-ba3e-878dacd4b823 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:31.669338 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8513446f-f004-4614-ba3e-878dacd4b823 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:31.670778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8513446f-f004-4614-ba3e-878dacd4b823 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:31.671555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8513446f-f004-4614-ba3e-878dacd4b823 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:31.750963 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c198dee1-7e68-442d-90b5-60d483beffde tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1515462211"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:31.752289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c198dee1-7e68-442d-90b5-60d483beffde tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:31.763839 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c198dee1-7e68-442d-90b5-60d483beffde tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:31.769876 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c198dee1-7e68-442d-90b5-60d483beffde tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:31.772786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c198dee1-7e68-442d-90b5-60d483beffde tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:31.774523 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c198dee1-7e68-442d-90b5-60d483beffde tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:31.778056 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c198dee1-7e68-442d-90b5-60d483beffde tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:31.779949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c198dee1-7e68-442d-90b5-60d483beffde tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:31.955651 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8513446f-f004-4614-ba3e-878dacd4b823 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:31.966020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8513446f-f004-4614-ba3e-878dacd4b823 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1678 microversion: 2.1 time: 0.314580 Oct 20 09:07:31.998657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 486/973] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:31 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1678 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:32.377073 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.network.security_group_api [None req-c198dee1-7e68-442d-90b5-60d483beffde tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Removing security group 8dd08cd5-03a3-4623-a8b2-414b5852ea61 from port 3c5f5de7-af1f-4408-ba7d-6be5f0344198 Oct 20 09:07:32.995834 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3d67521a-1a68-4ba2-9215-01da77381978 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:32.996458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3d67521a-1a68-4ba2-9215-01da77381978 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:33.003803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d67521a-1a68-4ba2-9215-01da77381978 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:33.004295 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d67521a-1a68-4ba2-9215-01da77381978 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:33.004730 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d67521a-1a68-4ba2-9215-01da77381978 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:33.005578 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d67521a-1a68-4ba2-9215-01da77381978 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:33.006176 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d67521a-1a68-4ba2-9215-01da77381978 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:33.006637 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3d67521a-1a68-4ba2-9215-01da77381978 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:33.072097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3d67521a-1a68-4ba2-9215-01da77381978 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:33.080744 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3d67521a-1a68-4ba2-9215-01da77381978 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1611 microversion: 2.1 time: 0.090338 Oct 20 09:07:33.081332 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 487/974] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:32 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1611 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:33.186685 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c198dee1-7e68-442d-90b5-60d483beffde tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "POST /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/action" status: 202 len: 0 microversion: 2.1 time: 1.440676 Oct 20 09:07:33.188790 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 488/975] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:07:31 2022] POST /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/action => generated 0 bytes in 1443 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:07:33.198794 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-1de3c2b5-9aae-4741-9ad9-161913978b82 req-cd9aba5c-7159-48a5-9c4c-4eb1b7eccd43 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03c3c04f-2850-480f-8d15-6f15327734e0", "tag": "3c5f5de7-af1f-4408-ba7d-6be5f0344198"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:33.211104 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-1de3c2b5-9aae-4741-9ad9-161913978b82 req-cd9aba5c-7159-48a5-9c4c-4eb1b7eccd43 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:33.211741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-1de3c2b5-9aae-4741-9ad9-161913978b82 req-cd9aba5c-7159-48a5-9c4c-4eb1b7eccd43 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:33.212233 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-1de3c2b5-9aae-4741-9ad9-161913978b82 req-cd9aba5c-7159-48a5-9c4c-4eb1b7eccd43 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:33.242453 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-1de3c2b5-9aae-4741-9ad9-161913978b82 req-cd9aba5c-7159-48a5-9c4c-4eb1b7eccd43 service nova] Creating event network-changed:3c5f5de7-af1f-4408-ba7d-6be5f0344198 for instance 03c3c04f-2850-480f-8d15-6f15327734e0 on hv-cinder-maste Oct 20 09:07:33.255066 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-1de3c2b5-9aae-4741-9ad9-161913978b82 req-cd9aba5c-7159-48a5-9c4c-4eb1b7eccd43 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060292 Oct 20 09:07:33.256919 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 488/976] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:07:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:34.104405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:34.105556 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:34.117098 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:34.117891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:34.119242 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:34.123032 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:34.123735 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:34.124532 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:34.210886 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:34.211532 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:34.211809 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:34.212269 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lazy-loading 'fault' on Instance uuid c5dba102-6a02-40f4-a502-d1fc1cdab301 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:07:34.319396 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8002ab26f6d64822aaa65aa730781b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fcf51428b6bd4c428e1e01aadc6abe48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:34.331963 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-55ddac0c-de4b-4f83-88c8-e28b58e00243 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 200 len: 1329 microversion: 2.1 time: 0.231242 Oct 20 09:07:34.333285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 489/977] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:34 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 1329 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:35.357999 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:35.358736 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:35.372319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:35.373185 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:35.373856 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:35.374834 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:35.381242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:35.382454 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:35.399990 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 HTTP/1.1 Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:35.403224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:35.404645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': 'c5dba102-6a02-40f4-a502-d1fc1cdab301'} Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:35.405427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:35.406729 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:35.410192 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:35.411725 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b5e0e184-8081-4f75-adf1-aac4adb81717 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 404 len: 111 microversion: 2.1 time: 0.058537 Oct 20 09:07:35.412568 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 489/978] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:35 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:35.422645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0b716793-625b-4908-9e4c-86297fd87305 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '.version_select at 0x7f5afb9e4430>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:35.447238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0b716793-625b-4908-9e4c-86297fd87305 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-437173038" status: 202 len: 0 microversion: 2.1 time: 0.029146 Oct 20 09:07:35.453094 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 490/979] 10.222.0.36 () {58 vars in 1218 bytes} [Thu Oct 20 09:07:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-437173038 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 20 09:07:35.541264 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-c2dcb0b7-a850-4e9c-a429-1f333bc82c07 req-572f809b-904e-4e92-9388-5ad07ace3231 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c5dba102-6a02-40f4-a502-d1fc1cdab301", "name": "network-vif-deleted", "tag": "a91a700e-d01e-4c84-8f53-4255f46188de"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:35.551137 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-354982a5-d2e9-4f5f-87a8-f05d66509b0f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:35.551675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-354982a5-d2e9-4f5f-87a8-f05d66509b0f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:35.564487 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-c2dcb0b7-a850-4e9c-a429-1f333bc82c07 req-572f809b-904e-4e92-9388-5ad07ace3231 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:35.564991 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-c2dcb0b7-a850-4e9c-a429-1f333bc82c07 req-572f809b-904e-4e92-9388-5ad07ace3231 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:35.565444 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-c2dcb0b7-a850-4e9c-a429-1f333bc82c07 req-572f809b-904e-4e92-9388-5ad07ace3231 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:35.575176 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-354982a5-d2e9-4f5f-87a8-f05d66509b0f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:35.575801 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-354982a5-d2e9-4f5f-87a8-f05d66509b0f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:35.576266 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-354982a5-d2e9-4f5f-87a8-f05d66509b0f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:35.577013 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-354982a5-d2e9-4f5f-87a8-f05d66509b0f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:35.577475 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-354982a5-d2e9-4f5f-87a8-f05d66509b0f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:35.584628 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-354982a5-d2e9-4f5f-87a8-f05d66509b0f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:35.606820 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-c2dcb0b7-a850-4e9c-a429-1f333bc82c07 req-572f809b-904e-4e92-9388-5ad07ace3231 service nova] Creating event network-vif-deleted:a91a700e-d01e-4c84-8f53-4255f46188de for instance c5dba102-6a02-40f4-a502-d1fc1cdab301 on hv-cinder-maste Oct 20 09:07:35.621268 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-c2dcb0b7-a850-4e9c-a429-1f333bc82c07 req-572f809b-904e-4e92-9388-5ad07ace3231 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083390 Oct 20 09:07:35.622044 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 490/980] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:07:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:35.648951 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-73b15154-5cb5-402b-8038-964c779be8fa req-dd04a7e6-f836-4dc2-829f-0a167450a26b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03c3c04f-2850-480f-8d15-6f15327734e0", "tag": "3c5f5de7-af1f-4408-ba7d-6be5f0344198"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:35.672264 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-73b15154-5cb5-402b-8038-964c779be8fa req-dd04a7e6-f836-4dc2-829f-0a167450a26b service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:35.679605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-73b15154-5cb5-402b-8038-964c779be8fa req-dd04a7e6-f836-4dc2-829f-0a167450a26b service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:35.680125 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-73b15154-5cb5-402b-8038-964c779be8fa req-dd04a7e6-f836-4dc2-829f-0a167450a26b service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:35.703252 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-73b15154-5cb5-402b-8038-964c779be8fa req-dd04a7e6-f836-4dc2-829f-0a167450a26b service nova] Creating event network-changed:3c5f5de7-af1f-4408-ba7d-6be5f0344198 for instance 03c3c04f-2850-480f-8d15-6f15327734e0 on hv-cinder-maste Oct 20 09:07:35.713978 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-73b15154-5cb5-402b-8038-964c779be8fa req-dd04a7e6-f836-4dc2-829f-0a167450a26b service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085832 Oct 20 09:07:35.715718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 491/981] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:07:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:35.796587 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-354982a5-d2e9-4f5f-87a8-f05d66509b0f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lazy-loading 'services' on Instance uuid 03c3c04f-2850-480f-8d15-6f15327734e0 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:07:35.944179 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-354982a5-d2e9-4f5f-87a8-f05d66509b0f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "DELETE /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments/e6fc3341-7c17-4457-942c-6564d78976e5" status: 202 len: 0 microversion: 2.1 time: 0.397841 Oct 20 09:07:35.948262 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 491/982] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:07:35 2022] DELETE /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments/e6fc3341-7c17-4457-942c-6564d78976e5 => generated 0 bytes in 399 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:07:35.956012 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-922ed609-df99-494f-aee9-cdaec880c423 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:35.958076 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-922ed609-df99-494f-aee9-cdaec880c423 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:35.976921 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-922ed609-df99-494f-aee9-cdaec880c423 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:35.978926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-922ed609-df99-494f-aee9-cdaec880c423 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:35.981536 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-922ed609-df99-494f-aee9-cdaec880c423 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:35.983572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-922ed609-df99-494f-aee9-cdaec880c423 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:35.984714 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-922ed609-df99-494f-aee9-cdaec880c423 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:35.986122 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-922ed609-df99-494f-aee9-cdaec880c423 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:36.031768 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-922ed609-df99-494f-aee9-cdaec880c423 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080093 Oct 20 09:07:36.032540 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 492/983] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:35 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:36.426105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8edbfd15-d304-4e39-8019-d25f6afd5a07 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Action: 'create', calling method: .version_select at 0x7f5afba2eee0>, body: {"keypair": {"name": "tempest-TestShelveInstance-1651321010", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB2tP4TjZTGMhdKpEV+Pyz1CPRbg9UPWLUncLtEtCbJdgyZM50/Rw6vYCap6kzmvbEbMxu7c7EpQaJRMQrNMwjiVkC+pwTSKVDAwTEJ8mKs6Om0tMlc6IWpy9+tEv7s7MQ=="}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:36.443733 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-8edbfd15-d304-4e39-8019-d25f6afd5a07 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Getting quotas for project 45d1505e085847329f7480c9bc08f1d8. Resources: {'key_pairs'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:07:36.449262 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-8edbfd15-d304-4e39-8019-d25f6afd5a07 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Getting quotas for user 21311bc5eef742f5a9eb81462a701ceb and project 45d1505e085847329f7480c9bc08f1d8. Resources: {'key_pairs'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:07:36.481812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8edbfd15-d304-4e39-8019-d25f6afd5a07 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.431218 Oct 20 09:07:36.490531 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 492/984] 10.222.0.36 () {60 vars in 1159 bytes} [Thu Oct 20 09:07:36 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:37.044959 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-659b860a-77fb-4375-9d16-cbf7f3575788 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:37.047259 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-659b860a-77fb-4375-9d16-cbf7f3575788 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:37.055391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-659b860a-77fb-4375-9d16-cbf7f3575788 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:37.055880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-659b860a-77fb-4375-9d16-cbf7f3575788 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:37.056312 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-659b860a-77fb-4375-9d16-cbf7f3575788 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:37.057012 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-659b860a-77fb-4375-9d16-cbf7f3575788 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:37.057493 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-659b860a-77fb-4375-9d16-cbf7f3575788 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:37.057993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-659b860a-77fb-4375-9d16-cbf7f3575788 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:37.111985 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-659b860a-77fb-4375-9d16-cbf7f3575788 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071498 Oct 20 09:07:37.114108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 493/985] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:37 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:37.865830 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:37.866626 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:37.878756 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:37.879539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:37.880111 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:37.881693 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:37.882883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:37.883564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:37.904780 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: DELETE /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 HTTP/1.1 Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:07:37.906851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:37.908360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': 'c5dba102-6a02-40f4-a502-d1fc1cdab301'} Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:37.909226 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:37.911265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:37.913466 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:37.913466 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e60c92b5-66e1-441a-9430-9dc4c542c5b0 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 404 len: 111 microversion: 2.1 time: 0.049942 Oct 20 09:07:37.913466 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 493/986] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:07:37 2022] DELETE /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:37.919833 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:37.920481 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: c5dba102-6a02-40f4-a502-d1fc1cdab301] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:37.936531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:37.937033 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:37.937555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:37.938352 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:37.938847 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:37.939270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:37.956190 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 HTTP/1.1 Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:37.958457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:37.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': 'c5dba102-6a02-40f4-a502-d1fc1cdab301'} Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:37.960682 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. Oct 20 09:07:37.961909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:37.963182 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance c5dba102-6a02-40f4-a502-d1fc1cdab301 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:37.963182 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d713e475-48e8-4ed3-886e-7acd0415e16d tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301" status: 404 len: 111 microversion: 2.1 time: 0.046714 Oct 20 09:07:37.965535 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 494/987] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:37 2022] GET /compute/v2.1/servers/c5dba102-6a02-40f4-a502-d1fc1cdab301 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:38.133387 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-fb0462c6-a570-452a-859b-6f2c4bfc75fb tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:38.135899 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-fb0462c6-a570-452a-859b-6f2c4bfc75fb tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:38.147795 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fb0462c6-a570-452a-859b-6f2c4bfc75fb tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:38.148578 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fb0462c6-a570-452a-859b-6f2c4bfc75fb tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:38.149178 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fb0462c6-a570-452a-859b-6f2c4bfc75fb tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:38.150065 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fb0462c6-a570-452a-859b-6f2c4bfc75fb tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:38.150542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fb0462c6-a570-452a-859b-6f2c4bfc75fb tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:38.152398 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-fb0462c6-a570-452a-859b-6f2c4bfc75fb tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:38.208254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-fb0462c6-a570-452a-859b-6f2c4bfc75fb tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080552 Oct 20 09:07:38.210731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 494/988] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:38 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:39.137837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fcae4f91-c2b3-4efd-8482-c27eb51f7ad3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:39.140058 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fcae4f91-c2b3-4efd-8482-c27eb51f7ad3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:39.148207 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fcae4f91-c2b3-4efd-8482-c27eb51f7ad3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.148699 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fcae4f91-c2b3-4efd-8482-c27eb51f7ad3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.149133 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fcae4f91-c2b3-4efd-8482-c27eb51f7ad3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.149836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fcae4f91-c2b3-4efd-8482-c27eb51f7ad3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.150322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fcae4f91-c2b3-4efd-8482-c27eb51f7ad3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.150788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fcae4f91-c2b3-4efd-8482-c27eb51f7ad3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.190811 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-fcae4f91-c2b3-4efd-8482-c27eb51f7ad3 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.057021 Oct 20 09:07:39.191396 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 495/989] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:39 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments => generated 371 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:39.221665 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e6b1af76-a01d-488a-b2df-76746bea5fdc tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:39.224697 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e6b1af76-a01d-488a-b2df-76746bea5fdc tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:39.232397 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6b1af76-a01d-488a-b2df-76746bea5fdc tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.232918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6b1af76-a01d-488a-b2df-76746bea5fdc tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.233367 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6b1af76-a01d-488a-b2df-76746bea5fdc tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.234136 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6b1af76-a01d-488a-b2df-76746bea5fdc tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.234691 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6b1af76-a01d-488a-b2df-76746bea5fdc tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.235164 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e6b1af76-a01d-488a-b2df-76746bea5fdc tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.268438 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e6b1af76-a01d-488a-b2df-76746bea5fdc tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.050663 Oct 20 09:07:39.269270 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 495/990] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:39 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 20 09:07:39.271561 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:39.272132 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:39.280276 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.282844 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:39.283189 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.283338 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:39.283954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.284834 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.285555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.285989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.289939 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.290423 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.290894 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.291612 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.292063 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.293018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.304998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: DELETE /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 HTTP/1.1 Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:39.306796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:07:39.308869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '06d8b443-0b7d-4287-8469-eb4a35863289'} Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:39.309796 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:07:39.314269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.315866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:39.315866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-482b2e18-4b16-4b69-9e28-aaf0d47ead14 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 404 len: 111 microversion: 2.1 time: 0.040657 Oct 20 09:07:39.315866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 496/991] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:07:39 2022] DELETE /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:39.321743 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:39.322361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: 06d8b443-0b7d-4287-8469-eb4a35863289] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:39.331785 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.332429 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.333096 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.334286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.334908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.335563 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.352851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 HTTP/1.1 Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:39.354572 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:07:39.357143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '06d8b443-0b7d-4287-8469-eb4a35863289'} Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:39.358205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. Oct 20 09:07:39.359717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.361577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance 06d8b443-0b7d-4287-8469-eb4a35863289 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:39.363018 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bdef352e-5791-4265-b870-b75c1e83058b tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289" status: 404 len: 111 microversion: 2.1 time: 0.043822 Oct 20 09:07:39.363018 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 497/992] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:39 2022] GET /compute/v2.1/servers/06d8b443-0b7d-4287-8469-eb4a35863289 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:39.368722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7000a083-a973-4085-abce-bf49e4e79e6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:39.369356 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7000a083-a973-4085-abce-bf49e4e79e6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:39.376494 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7000a083-a973-4085-abce-bf49e4e79e6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.377248 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7000a083-a973-4085-abce-bf49e4e79e6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.377909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7000a083-a973-4085-abce-bf49e4e79e6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.378874 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7000a083-a973-4085-abce-bf49e4e79e6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.379567 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7000a083-a973-4085-abce-bf49e4e79e6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.380206 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7000a083-a973-4085-abce-bf49e4e79e6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.396668 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] HTTP exception thrown: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 is not attached to volume e6fc3341-7c17-4457-942c-6564d78976e5 Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Request method failure captured: Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: DELETE /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments/e6fc3341-7c17-4457-942c-6564d78976e5 HTTP/1.1 Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ9DkOnCPmnFPAYepq2loImDxYdNr5cwNuK0FipLPZmtXhqlKuDFY5HfbRZ8DMteGiPseX8yjuTJWmYc2b3iCsNPJMR1z0rUaFXEopQsT81Rl7L5mAg-N_9p2t6IMAHa6c8ew6fROX3yi9quklri0gQb_DmLFnvmFFkO56rdzMKEeI4 Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 020d4019643f4499b5162717fbf7b5b5 Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-TestMinimumBasicScenario-1711789739 Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:07:39.399169 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/020d4019643f4499b5162717fbf7b5b5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_020d4019643f4499b5162717fbf7b5b5", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/020d4019643f4499b5162717fbf7b5b5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/020d4019643f4499b5162717fbf7b5b5"}]}] Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-TestMinimumBasicScenario-1711789739 Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 020d4019643f4499b5162717fbf7b5b5 Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1711789739 Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-TestMinimumBasicScenario-1711789739-project Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 14ddd15ab87a4d0fbea1d6dfa910f8ce Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-TestMinimumBasicScenario-1711789739-project Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:07:39.400767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 is not attached to volume e6fc3341-7c17-4457-942c-6564d78976e5 Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'server_id': '03c3c04f-2850-480f-8d15-6f15327734e0', 'id': 'e6fc3341-7c17-4457-942c-6564d78976e5'} Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id e6fc3341-7c17-4457-942c-6564d78976e5. Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:39.401704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 is not attached to volume e6fc3341-7c17-4457-942c-6564d78976e5 Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 is not attached to volume e6fc3341-7c17-4457-942c-6564d78976e5 Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.403840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Returning 404 to user: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 is not attached to volume e6fc3341-7c17-4457-942c-6564d78976e5 {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:39.404848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bee57432-a641-465f-bcb2-05160e67ac6f tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "DELETE /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments/e6fc3341-7c17-4457-942c-6564d78976e5" status: 404 len: 154 microversion: 2.1 time: 0.125010 Oct 20 09:07:39.404848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 496/993] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:07:39 2022] DELETE /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0/os-volume_attachments/e6fc3341-7c17-4457-942c-6564d78976e5 => generated 154 bytes in 126 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:39.410374 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:39.415141 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:39.422920 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.423506 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.424024 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.424823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.425344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.425629 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7000a083-a973-4085-abce-bf49e4e79e6c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments/b1034f78-9144-40e6-aa02-158b3e3d51e5" status: 200 len: 194 microversion: 2.1 time: 0.060828 Oct 20 09:07:39.425984 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.427028 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 498/994] 10.222.0.36 () {58 vars in 1413 bytes} [Thu Oct 20 09:07:39 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments/b1034f78-9144-40e6-aa02-158b3e3d51e5 => generated 194 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:39.440876 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-379b45ed-8a07-4d2c-b902-000f67a1fe79 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:39.441957 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-379b45ed-8a07-4d2c-b902-000f67a1fe79 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:39.449306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-379b45ed-8a07-4d2c-b902-000f67a1fe79 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.449835 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-379b45ed-8a07-4d2c-b902-000f67a1fe79 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.450294 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-379b45ed-8a07-4d2c-b902-000f67a1fe79 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.451255 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-379b45ed-8a07-4d2c-b902-000f67a1fe79 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.451723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-379b45ed-8a07-4d2c-b902-000f67a1fe79 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.451997 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:07:39.452124 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-379b45ed-8a07-4d2c-b902-000f67a1fe79 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: DELETE /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 HTTP/1.1 Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:07:39.453567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:07:39.455195 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': 'aae7b08a-4c9a-4d12-9598-193afab4ca00'} Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:39.456394 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:07:39.458598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.460112 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:39.460112 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-86abafb6-db6e-43cf-9bfd-e5ac7f67fc01 tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 404 len: 111 microversion: 2.1 time: 0.052431 Oct 20 09:07:39.460112 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 497/995] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:07:39 2022] DELETE /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:39.468927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:39.469542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: aae7b08a-4c9a-4d12-9598-193afab4ca00] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:39.478359 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.479406 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.480544 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.481754 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:39.482355 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:39.483035 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:39.503409 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 HTTP/1.1 Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:07:39.505023 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:07:39.506772 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': 'aae7b08a-4c9a-4d12-9598-193afab4ca00'} Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:39.507630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. Oct 20 09:07:39.509124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:39.510893 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance aae7b08a-4c9a-4d12-9598-193afab4ca00 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:39.510893 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f17c4f98-442e-49f2-9884-5ca84571314a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00" status: 404 len: 111 microversion: 2.1 time: 0.041738 Oct 20 09:07:39.510893 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 498/996] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:39 2022] GET /compute/v2.1/servers/aae7b08a-4c9a-4d12-9598-193afab4ca00 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:39.560776 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-379b45ed-8a07-4d2c-b902-000f67a1fe79 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lazy-loading 'services' on Instance uuid 814ac79f-b35d-4aac-9b42-9f4ae8a83322 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:07:39.641089 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-379b45ed-8a07-4d2c-b902-000f67a1fe79 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "DELETE /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments/b1034f78-9144-40e6-aa02-158b3e3d51e5" status: 202 len: 0 microversion: 2.1 time: 0.205488 Oct 20 09:07:39.641884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 499/997] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:07:39 2022] DELETE /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments/b1034f78-9144-40e6-aa02-158b3e3d51e5 => generated 0 bytes in 208 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:07:40.582352 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-dbc0111b-2f54-49f4-92f6-719371089643 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:40.583084 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-dbc0111b-2f54-49f4-92f6-719371089643 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:40.589903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dbc0111b-2f54-49f4-92f6-719371089643 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:40.591134 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dbc0111b-2f54-49f4-92f6-719371089643 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:40.591878 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dbc0111b-2f54-49f4-92f6-719371089643 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:40.592941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dbc0111b-2f54-49f4-92f6-719371089643 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:40.593561 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dbc0111b-2f54-49f4-92f6-719371089643 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:40.594163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-dbc0111b-2f54-49f4-92f6-719371089643 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:40.626289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-dbc0111b-2f54-49f4-92f6-719371089643 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:07:40.628400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:07:40.685910 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:40.687226 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:40.694865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:40.695348 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:40.695774 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:40.696473 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:40.696917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:40.697334 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:40.712055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-dbc0111b-2f54-49f4-92f6-719371089643 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "DELETE /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 204 len: 0 microversion: 2.1 time: 0.133421 Oct 20 09:07:40.712306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:07:40.713009 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 499/998] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:07:40 2022] DELETE /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: DELETE /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 HTTP/1.1 Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:40.713979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:07:40.715926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': 'ec9ae57d-3e5a-490f-beba-d071e86a7b31'} Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:40.716993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:07:40.719032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:40.721803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:40.722816 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-97ad0795-9104-409e-86ef-c175f39b051c tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "DELETE /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 404 len: 111 microversion: 2.1 time: 0.042974 Oct 20 09:07:40.723664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 500/999] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:07:40 2022] DELETE /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:40.729381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ea742906-fab8-4c84-b158-185d91bc7111 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:40.730090 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ea742906-fab8-4c84-b158-185d91bc7111 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:40.733322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:40.734177 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] [instance: ec9ae57d-3e5a-490f-beba-d071e86a7b31] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:40.736702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ea742906-fab8-4c84-b158-185d91bc7111 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:40.737555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ea742906-fab8-4c84-b158-185d91bc7111 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:40.737983 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ea742906-fab8-4c84-b158-185d91bc7111 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:40.738716 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ea742906-fab8-4c84-b158-185d91bc7111 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:40.739171 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ea742906-fab8-4c84-b158-185d91bc7111 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:40.739599 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ea742906-fab8-4c84-b158-185d91bc7111 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:40.740436 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:40.740914 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:40.741345 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:40.742036 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:40.742499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:40.742920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:40.757808 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] HTTP exception thrown: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Request method failure captured: Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 HTTP/1.1 Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ48dt_dsT4ZHXvSrNyJoxhpOMpMNbkAN4ail1S6lmKhmAqAF49YPtst9MLF4b-vHhtpi2ZNp29YZtP8t2q92g1CkO5cNXxqzmZwRIP63XoAYvXNUx9QBdgiETwtChjKagSeYdr4gA_ZfkepzJ386yFy8Y7Dz-iM3nrItr5JgnoBp-E Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:40.759382 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_fcf51428b6bd4c428e1e01aadc6abe48", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/fcf51428b6bd4c428e1e01aadc6abe48"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/fcf51428b6bd4c428e1e01aadc6abe48"}]}] Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: fcf51428b6bd4c428e1e01aadc6abe48 Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1404176933 Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 8002ab26f6d64822aaa65aa730781b76 Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeNegativeTest-1404176933-project Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:07:40.760734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': 'ec9ae57d-3e5a-490f-beba-d071e86a7b31'} Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:40.765234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. Oct 20 09:07:40.768857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:40.771265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] Returning 404 to user: Instance ec9ae57d-3e5a-490f-beba-d071e86a7b31 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:40.771265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-9a188dee-b632-4dd2-8771-6b412aa9490a tempest-AttachVolumeNegativeTest-1404176933 tempest-AttachVolumeNegativeTest-1404176933-project] 10.222.0.36 "GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31" status: 404 len: 111 microversion: 2.1 time: 0.033355 Oct 20 09:07:40.771265 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 501/1000] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:40 2022] GET /compute/v2.1/servers/ec9ae57d-3e5a-490f-beba-d071e86a7b31 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 20 09:07:40.884854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ea742906-fab8-4c84-b158-185d91bc7111 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:40.894809 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ea742906-fab8-4c84-b158-185d91bc7111 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1555 microversion: 2.1 time: 0.170313 Oct 20 09:07:40.895884 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 500/1001] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:40 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1555 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:41.922769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-55f720de-edb2-407a-9fdc-514c201d768c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:41.923459 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-55f720de-edb2-407a-9fdc-514c201d768c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:41.925924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=72,nova_api:UPDATE=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:07:41.927523 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4784786c-ffdb-4722-a74d-1fc1a7e52d78 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:41.930657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4784786c-ffdb-4722-a74d-1fc1a7e52d78 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:41.947481 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4784786c-ffdb-4722-a74d-1fc1a7e52d78 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:41.950327 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4784786c-ffdb-4722-a74d-1fc1a7e52d78 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:41.953797 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4784786c-ffdb-4722-a74d-1fc1a7e52d78 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:41.954781 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-55f720de-edb2-407a-9fdc-514c201d768c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:41.956564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4784786c-ffdb-4722-a74d-1fc1a7e52d78 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:41.957045 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4784786c-ffdb-4722-a74d-1fc1a7e52d78 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:41.957763 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-55f720de-edb2-407a-9fdc-514c201d768c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:41.958202 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-55f720de-edb2-407a-9fdc-514c201d768c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:41.958946 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-55f720de-edb2-407a-9fdc-514c201d768c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:41.959395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-55f720de-edb2-407a-9fdc-514c201d768c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:41.959815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-55f720de-edb2-407a-9fdc-514c201d768c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:41.960561 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4784786c-ffdb-4722-a74d-1fc1a7e52d78 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:42.001657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:07:42.059683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-55f720de-edb2-407a-9fdc-514c201d768c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments/f409e45c-c769-41e7-bc65-2b8aefe5d85e" status: 200 len: 194 microversion: 2.1 time: 0.153010 Oct 20 09:07:42.061028 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 502/1002] 10.222.0.36 () {58 vars in 1413 bytes} [Thu Oct 20 09:07:41 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments/f409e45c-c769-41e7-bc65-2b8aefe5d85e => generated 194 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:42.073404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-db5a47bf-61e3-4749-8c87-39e4e5556389 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:42.073899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-db5a47bf-61e3-4749-8c87-39e4e5556389 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:42.082408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db5a47bf-61e3-4749-8c87-39e4e5556389 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:42.083388 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db5a47bf-61e3-4749-8c87-39e4e5556389 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:42.084304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db5a47bf-61e3-4749-8c87-39e4e5556389 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:42.085242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db5a47bf-61e3-4749-8c87-39e4e5556389 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:42.086340 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db5a47bf-61e3-4749-8c87-39e4e5556389 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:42.087047 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-db5a47bf-61e3-4749-8c87-39e4e5556389 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:42.253472 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-db5a47bf-61e3-4749-8c87-39e4e5556389 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lazy-loading 'services' on Instance uuid 814ac79f-b35d-4aac-9b42-9f4ae8a83322 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:07:42.264555 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4784786c-ffdb-4722-a74d-1fc1a7e52d78 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:42.285791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4784786c-ffdb-4722-a74d-1fc1a7e52d78 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1555 microversion: 2.1 time: 0.366279 Oct 20 09:07:42.303308 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 501/1003] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:41 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1555 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:42.336889 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-db5a47bf-61e3-4749-8c87-39e4e5556389 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "DELETE /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments/f409e45c-c769-41e7-bc65-2b8aefe5d85e" status: 202 len: 0 microversion: 2.1 time: 0.267666 Oct 20 09:07:42.337590 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 503/1004] 10.222.0.36 () {58 vars in 1416 bytes} [Thu Oct 20 09:07:42 2022] DELETE /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments/f409e45c-c769-41e7-bc65-2b8aefe5d85e => generated 0 bytes in 269 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:07:43.313624 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9d52ff7e-6875-4f50-83b1-d6605ab3c0a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:43.314889 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-9d52ff7e-6875-4f50-83b1-d6605ab3c0a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:43.331033 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d52ff7e-6875-4f50-83b1-d6605ab3c0a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:43.337336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d52ff7e-6875-4f50-83b1-d6605ab3c0a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:43.338001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d52ff7e-6875-4f50-83b1-d6605ab3c0a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:43.339423 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d52ff7e-6875-4f50-83b1-d6605ab3c0a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:43.342413 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d52ff7e-6875-4f50-83b1-d6605ab3c0a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:43.343149 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9d52ff7e-6875-4f50-83b1-d6605ab3c0a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:43.586716 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-9d52ff7e-6875-4f50-83b1-d6605ab3c0a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:43.595786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-9d52ff7e-6875-4f50-83b1-d6605ab3c0a8 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1555 microversion: 2.1 time: 0.288504 Oct 20 09:07:43.596598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 502/1005] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:43 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1555 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:44.364554 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-020c062c-2ff3-4116-b963-3e6e8314b84b req-d9320729-46d9-4e79-91f7-c7bb24dd51b7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "03c3c04f-2850-480f-8d15-6f15327734e0", "name": "network-vif-deleted", "tag": "3c5f5de7-af1f-4408-ba7d-6be5f0344198"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:44.392136 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-020c062c-2ff3-4116-b963-3e6e8314b84b req-d9320729-46d9-4e79-91f7-c7bb24dd51b7 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:44.392749 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-020c062c-2ff3-4116-b963-3e6e8314b84b req-d9320729-46d9-4e79-91f7-c7bb24dd51b7 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:44.393689 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-020c062c-2ff3-4116-b963-3e6e8314b84b req-d9320729-46d9-4e79-91f7-c7bb24dd51b7 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:44.422320 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-020c062c-2ff3-4116-b963-3e6e8314b84b req-d9320729-46d9-4e79-91f7-c7bb24dd51b7 service nova] Creating event network-vif-deleted:3c5f5de7-af1f-4408-ba7d-6be5f0344198 for instance 03c3c04f-2850-480f-8d15-6f15327734e0 on hv-cinder-maste Oct 20 09:07:44.437611 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-020c062c-2ff3-4116-b963-3e6e8314b84b req-d9320729-46d9-4e79-91f7-c7bb24dd51b7 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079929 Oct 20 09:07:44.439106 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 504/1006] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:07:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:44.626347 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:44.627102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:44.635714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:44.636204 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:44.636668 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:44.637897 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:44.639002 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:44.639428 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:44.698461 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:44.699081 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:44.699517 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:44.700111 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lazy-loading 'fault' on Instance uuid 03c3c04f-2850-480f-8d15-6f15327734e0 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:07:44.708907 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8d093132-6477-4e0e-a6a7-08f1a324b7d5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:44.730353 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8d093132-6477-4e0e-a6a7-08f1a324b7d5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:44.744845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8d093132-6477-4e0e-a6a7-08f1a324b7d5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:44.749301 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8d093132-6477-4e0e-a6a7-08f1a324b7d5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:44.750453 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8d093132-6477-4e0e-a6a7-08f1a324b7d5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:44.754351 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8d093132-6477-4e0e-a6a7-08f1a324b7d5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:44.757361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8d093132-6477-4e0e-a6a7-08f1a324b7d5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:44.757361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8d093132-6477-4e0e-a6a7-08f1a324b7d5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:44.806382 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:44.830024 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-58619928-09f0-495b-9757-c80ec6941304 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1349 microversion: 2.1 time: 0.212879 Oct 20 09:07:44.833707 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 503/1007] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:44 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1349 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:44.842781 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8d093132-6477-4e0e-a6a7-08f1a324b7d5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.139666 Oct 20 09:07:44.850254 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 505/1008] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:44 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments => generated 25 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 20 09:07:44.997996 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6894e819-e4ea-4b03-9bcf-b8080f9d9011 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:45.000849 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6894e819-e4ea-4b03-9bcf-b8080f9d9011 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:45.006832 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6894e819-e4ea-4b03-9bcf-b8080f9d9011 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:45.007374 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6894e819-e4ea-4b03-9bcf-b8080f9d9011 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:45.007813 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6894e819-e4ea-4b03-9bcf-b8080f9d9011 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:45.008532 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6894e819-e4ea-4b03-9bcf-b8080f9d9011 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:45.009108 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6894e819-e4ea-4b03-9bcf-b8080f9d9011 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:45.009533 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6894e819-e4ea-4b03-9bcf-b8080f9d9011 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:45.039984 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6894e819-e4ea-4b03-9bcf-b8080f9d9011 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046061 Oct 20 09:07:45.040461 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 504/1009] 10.222.0.36 () {58 vars in 1302 bytes} [Thu Oct 20 09:07:44 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 20 09:07:45.112245 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-c853e1be-e483-43c9-bbb9-278ab87e70be tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:45.112763 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c853e1be-e483-43c9-bbb9-278ab87e70be tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:45.118662 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c853e1be-e483-43c9-bbb9-278ab87e70be tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:45.119146 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c853e1be-e483-43c9-bbb9-278ab87e70be tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:45.119581 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c853e1be-e483-43c9-bbb9-278ab87e70be tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:45.120277 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c853e1be-e483-43c9-bbb9-278ab87e70be tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:45.120738 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c853e1be-e483-43c9-bbb9-278ab87e70be tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:45.121244 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-c853e1be-e483-43c9-bbb9-278ab87e70be tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:45.147634 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-c853e1be-e483-43c9-bbb9-278ab87e70be tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Going to try to terminate instance {{(pid=119067) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:07:45.235515 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-c853e1be-e483-43c9-bbb9-278ab87e70be tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "DELETE /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 204 len: 0 microversion: 2.1 time: 0.126681 Oct 20 09:07:45.238341 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 506/1010] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:07:45 2022] DELETE /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:07:45.244701 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-43d82932-3c81-4d17-a513-b298b1287f9c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:45.253714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-43d82932-3c81-4d17-a513-b298b1287f9c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:45.269151 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43d82932-3c81-4d17-a513-b298b1287f9c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:45.270621 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43d82932-3c81-4d17-a513-b298b1287f9c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:45.271564 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43d82932-3c81-4d17-a513-b298b1287f9c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:45.272739 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43d82932-3c81-4d17-a513-b298b1287f9c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:45.273594 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43d82932-3c81-4d17-a513-b298b1287f9c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:45.274225 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-43d82932-3c81-4d17-a513-b298b1287f9c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:45.414799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-43d82932-3c81-4d17-a513-b298b1287f9c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:45.423684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-43d82932-3c81-4d17-a513-b298b1287f9c tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1674 microversion: 2.1 time: 0.182522 Oct 20 09:07:45.424293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 505/1011] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:45 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1674 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:45.859592 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:45.860365 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:45.868033 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:45.868696 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:45.869142 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:45.869822 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:45.870269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:45.870775 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:45.907342 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:45.907833 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:45.908301 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:45.908755 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lazy-loading 'fault' on Instance uuid 03c3c04f-2850-480f-8d15-6f15327734e0 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:07:45.952273 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:45.960631 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-85aa10fb-a84b-418b-b6cb-7079d7713d74 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1349 microversion: 2.1 time: 0.106344 Oct 20 09:07:45.965522 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 507/1012] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:45 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1349 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:45.997499 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cb2fd163-cd49-427e-a63c-9a914de0b5d0 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Action: 'create', calling method: .version_select at 0x7f5afbb0c9d0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-249552947", "metadata": {"Type": "work"}}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cb2fd163-cd49-427e-a63c-9a914de0b5d0 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Request method failure captured: Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 121 Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ_h1Yj1GrP2pUjitXxiQRC_bqSP5bAQsWwAX2pCYi7ERdtdkKpdDf4XGk50XdxymP4P6UmhASEkR1yV2uYYZFAaJ1Ds3NisMeToUxXK7IoG1DSOHc1X1WU-1nHz6uj0-AiXK6CYM90EAFC8aLFo4TZ7POymeLnVp6refhUCQzdPVPc Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:07:46.009278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_4a8df23cfe9b4c779974847098bb6a8c", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}] Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 1c11f96690f04580be2c330b3f57dcc3 Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-249552947", "metadata": {"Type": "work"}}} Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: Oct 20 09:07:46.010933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-249552947', 'metadata': {'Type': 'work'}}}} Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise error Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi '#$%' Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:46.011930 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 20 09:07:46.014138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 20 09:07:46.015542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 20 09:07:46.015542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.015542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cb2fd163-cd49-427e-a63c-9a914de0b5d0 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:46.015542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-cb2fd163-cd49-427e-a63c-9a914de0b5d0 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] 10.222.0.36 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.131654 Oct 20 09:07:46.015542 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 506/1013] 10.222.0.36 () {60 vars in 1156 bytes} [Thu Oct 20 09:07:45 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 133 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 20 09:07:46.021602 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-229625af-be41-4320-905e-83ee2de63ac5 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Action: 'create', calling method: .version_select at 0x7f5afb9d3790>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1688385958", "metadata": {"Type": "work"}}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-229625af-be41-4320-905e-83ee2de63ac5 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Request method failure captured: Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 120 Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ_h1Yj1GrP2pUjitXxiQRC_bqSP5bAQsWwAX2pCYi7ERdtdkKpdDf4XGk50XdxymP4P6UmhASEkR1yV2uYYZFAaJ1Ds3NisMeToUxXK7IoG1DSOHc1X1WU-1nHz6uj0-AiXK6CYM90EAFC8aLFo4TZ7POymeLnVp6refhUCQzdPVPc Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:46.026290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_4a8df23cfe9b4c779974847098bb6a8c", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}] Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 1c11f96690f04580be2c330b3f57dcc3 Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1688385958", "metadata": {"Type": "work"}}} Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: Oct 20 09:07:46.027908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1688385958', 'metadata': {'Type': 'work'}}}} Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise error Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi '0' Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:46.029381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 20 09:07:46.031020 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 20 09:07:46.032242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 20 09:07:46.032242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.032242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-229625af-be41-4320-905e-83ee2de63ac5 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:46.032242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-229625af-be41-4320-905e-83ee2de63ac5 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] 10.222.0.36 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.010363 Oct 20 09:07:46.032242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 508/1014] 10.222.0.36 () {60 vars in 1156 bytes} [Thu Oct 20 09:07:46 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 20 09:07:46.038143 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b43459e9-96d8-4780-b571-190215b9d766 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Action: 'create', calling method: .version_select at 0x7f5afba2e700>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-2071516085", "metadata": {"Type": "work"}}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b43459e9-96d8-4780-b571-190215b9d766 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Request method failure captured: Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 119 Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ_h1Yj1GrP2pUjitXxiQRC_bqSP5bAQsWwAX2pCYi7ERdtdkKpdDf4XGk50XdxymP4P6UmhASEkR1yV2uYYZFAaJ1Ds3NisMeToUxXK7IoG1DSOHc1X1WU-1nHz6uj0-AiXK6CYM90EAFC8aLFo4TZ7POymeLnVp6refhUCQzdPVPc Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:07:46.042840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_4a8df23cfe9b4c779974847098bb6a8c", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}] Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 1c11f96690f04580be2c330b3f57dcc3 Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-2071516085", "metadata": {"Type": "work"}}} Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: Oct 20 09:07:46.044891 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-2071516085', 'metadata': {'Type': 'work'}}}} Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise error Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi '' Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:46.045888 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 20 09:07:46.047552 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 20 09:07:46.048743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 20 09:07:46.048743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.048743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b43459e9-96d8-4780-b571-190215b9d766 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:46.048743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b43459e9-96d8-4780-b571-190215b9d766 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] 10.222.0.36 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.009887 Oct 20 09:07:46.048743 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 507/1015] 10.222.0.36 () {60 vars in 1156 bytes} [Thu Oct 20 09:07:46 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 20 09:07:46.054415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-54b2c608-d41b-405e-831d-6f7f45830d4c tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Calling method '.version_select at 0x7f5afbb283a0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:46.085670 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-54b2c608-d41b-405e-831d-6f7f45830d4c tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] HTTP exception thrown: Volume tempest-invalid-54314768 could not be found. Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-54b2c608-d41b-405e-831d-6f7f45830d4c tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Request method failure captured: Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-54314768 HTTP/1.1 Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ_h1Yj1GrP2pUjitXxiQRC_bqSP5bAQsWwAX2pCYi7ERdtdkKpdDf4XGk50XdxymP4P6UmhASEkR1yV2uYYZFAaJ1Ds3NisMeToUxXK7IoG1DSOHc1X1WU-1nHz6uj0-AiXK6CYM90EAFC8aLFo4TZ7POymeLnVp6refhUCQzdPVPc Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:46.087679 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_4a8df23cfe9b4c779974847098bb6a8c", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}] Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 1c11f96690f04580be2c330b3f57dcc3 Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Volume tempest-invalid-54314768 could not be found. Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': 'tempest-invalid-54314768'} Oct 20 09:07:46.089339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-54314768 could not be found. (HTTP 404) (Request-ID: req-ac4311a0-6383-4db8-be40-6dc646dcd188) Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.090939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 20 09:07:46.092534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-54314768 could not be found. Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:46.093824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-54314768 could not be found. Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-54314768 could not be found. Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-54b2c608-d41b-405e-831d-6f7f45830d4c tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Returning 404 to user: Volume tempest-invalid-54314768 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-54b2c608-d41b-405e-831d-6f7f45830d4c tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] 10.222.0.36 "DELETE /compute/v2.1/os-volumes/tempest-invalid-54314768" status: 404 len: 97 microversion: 2.1 time: 0.038689 Oct 20 09:07:46.095519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 509/1016] 10.222.0.36 () {58 vars in 1212 bytes} [Thu Oct 20 09:07:46 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-54314768 => generated 97 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 20 09:07:46.100567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5c403e19-2b2e-42f1-a0e8-3bb0182e0bb7 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] 10.222.0.36 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003808 Oct 20 09:07:46.101106 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 508/1017] 10.222.0.36 () {58 vars in 1140 bytes} [Thu Oct 20 09:07:46 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 20 09:07:46.110289 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f24de4a2-7e72-447b-8549-74cb5f209883 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004331 Oct 20 09:07:46.110805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 510/1018] 10.222.0.36 () {58 vars in 1137 bytes} [Thu Oct 20 09:07:46 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 20 09:07:46.119348 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-78eb9436-27c1-420d-aa04-4177c5942f11 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Calling method '.version_select at 0x7f5afbb0cca0>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:46.149102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-78eb9436-27c1-420d-aa04-4177c5942f11 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] HTTP exception thrown: Volume 123d42db-0eda-4926-b8dc-5e8847c972f2 could not be found. Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-78eb9436-27c1-420d-aa04-4177c5942f11 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Request method failure captured: Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: DELETE /compute/v2.1/os-volumes/123d42db-0eda-4926-b8dc-5e8847c972f2 HTTP/1.1 Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ_h1Yj1GrP2pUjitXxiQRC_bqSP5bAQsWwAX2pCYi7ERdtdkKpdDf4XGk50XdxymP4P6UmhASEkR1yV2uYYZFAaJ1Ds3NisMeToUxXK7IoG1DSOHc1X1WU-1nHz6uj0-AiXK6CYM90EAFC8aLFo4TZ7POymeLnVp6refhUCQzdPVPc Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:07:46.152434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_4a8df23cfe9b4c779974847098bb6a8c", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}] Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 1c11f96690f04580be2c330b3f57dcc3 Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Volume 123d42db-0eda-4926-b8dc-5e8847c972f2 could not be found. Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '123d42db-0eda-4926-b8dc-5e8847c972f2'} Oct 20 09:07:46.154114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 123d42db-0eda-4926-b8dc-5e8847c972f2 could not be found. (HTTP 404) (Request-ID: req-212a9b58-21b3-4977-af9c-9b0297df4776) Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.155221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 20 09:07:46.156580 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 123d42db-0eda-4926-b8dc-5e8847c972f2 could not be found. Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 20 09:07:46.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 123d42db-0eda-4926-b8dc-5e8847c972f2 could not be found. Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 123d42db-0eda-4926-b8dc-5e8847c972f2 could not be found. Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-78eb9436-27c1-420d-aa04-4177c5942f11 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Returning 404 to user: Volume 123d42db-0eda-4926-b8dc-5e8847c972f2 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-78eb9436-27c1-420d-aa04-4177c5942f11 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] 10.222.0.36 "DELETE /compute/v2.1/os-volumes/123d42db-0eda-4926-b8dc-5e8847c972f2" status: 404 len: 109 microversion: 2.1 time: 0.038432 Oct 20 09:07:46.159430 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 509/1019] 10.222.0.36 () {58 vars in 1248 bytes} [Thu Oct 20 09:07:46 2022] DELETE /compute/v2.1/os-volumes/123d42db-0eda-4926-b8dc-5e8847c972f2 => generated 109 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:46.163740 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-69638318-0388-442a-9ebc-555a181f9531 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Calling method '.version_select at 0x7f5afb9e4b80>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:46.195424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-69638318-0388-442a-9ebc-555a181f9531 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] HTTP exception thrown: Volume d74e770a-c84f-4b55-b0de-c75044ca06f8 could not be found. Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-69638318-0388-442a-9ebc-555a181f9531 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Request method failure captured: Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/os-volumes/d74e770a-c84f-4b55-b0de-c75044ca06f8 HTTP/1.1 Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ_h1Yj1GrP2pUjitXxiQRC_bqSP5bAQsWwAX2pCYi7ERdtdkKpdDf4XGk50XdxymP4P6UmhASEkR1yV2uYYZFAaJ1Ds3NisMeToUxXK7IoG1DSOHc1X1WU-1nHz6uj0-AiXK6CYM90EAFC8aLFo4TZ7POymeLnVp6refhUCQzdPVPc Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:46.197364 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_4a8df23cfe9b4c779974847098bb6a8c", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/4a8df23cfe9b4c779974847098bb6a8c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/4a8df23cfe9b4c779974847098bb6a8c"}]}] Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 4a8df23cfe9b4c779974847098bb6a8c Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-VolumesNegativeTest-1061801 Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 1c11f96690f04580be2c330b3f57dcc3 Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-VolumesNegativeTest-1061801-project Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Volume d74e770a-c84f-4b55-b0de-c75044ca06f8 could not be found. Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': 'd74e770a-c84f-4b55-b0de-c75044ca06f8'} Oct 20 09:07:46.198954 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume d74e770a-c84f-4b55-b0de-c75044ca06f8 could not be found. (HTTP 404) (Request-ID: req-2edc682b-0b75-4daa-aaa9-2e2b943e0159) Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.199969 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 20 09:07:46.201361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume d74e770a-c84f-4b55-b0de-c75044ca06f8 could not be found. Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 20 09:07:46.202899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume d74e770a-c84f-4b55-b0de-c75044ca06f8 could not be found. Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume d74e770a-c84f-4b55-b0de-c75044ca06f8 could not be found. Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-69638318-0388-442a-9ebc-555a181f9531 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] Returning 404 to user: Volume d74e770a-c84f-4b55-b0de-c75044ca06f8 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-69638318-0388-442a-9ebc-555a181f9531 tempest-VolumesNegativeTest-1061801 tempest-VolumesNegativeTest-1061801-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/d74e770a-c84f-4b55-b0de-c75044ca06f8" status: 404 len: 109 microversion: 2.1 time: 0.038825 Oct 20 09:07:46.204267 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 511/1020] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:07:46 2022] GET /compute/v2.1/os-volumes/d74e770a-c84f-4b55-b0de-c75044ca06f8 => generated 109 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:46.447178 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-765023eb-43c4-4c19-9e50-2ebc62053ee9 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:46.448396 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-765023eb-43c4-4c19-9e50-2ebc62053ee9 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:46.455745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-765023eb-43c4-4c19-9e50-2ebc62053ee9 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:46.456259 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-765023eb-43c4-4c19-9e50-2ebc62053ee9 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:46.456732 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-765023eb-43c4-4c19-9e50-2ebc62053ee9 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:46.457468 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-765023eb-43c4-4c19-9e50-2ebc62053ee9 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:46.457904 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-765023eb-43c4-4c19-9e50-2ebc62053ee9 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:46.458330 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-765023eb-43c4-4c19-9e50-2ebc62053ee9 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:46.458933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=4 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:07:46.607922 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-765023eb-43c4-4c19-9e50-2ebc62053ee9 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:46.615932 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-765023eb-43c4-4c19-9e50-2ebc62053ee9 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1674 microversion: 2.1 time: 0.172686 Oct 20 09:07:46.616501 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 510/1021] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:46 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1674 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:46.986605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:46.987261 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:46.993209 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:46.993698 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:46.994135 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:46.994856 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:46.995309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:46.995740 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:47.029253 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:47.029741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:47.030178 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:47.030658 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lazy-loading 'fault' on Instance uuid 03c3c04f-2850-480f-8d15-6f15327734e0 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:07:47.072395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '14ddd15ab87a4d0fbea1d6dfa910f8ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '020d4019643f4499b5162717fbf7b5b5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:47.081575 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-66948ee4-0214-4e0d-a288-0da3979020a2 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 200 len: 1349 microversion: 2.1 time: 0.100510 Oct 20 09:07:47.082302 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 512/1022] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:46 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 1349 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:47.644363 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1491ecc7-5513-4232-babe-143389a919c2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:47.645093 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1491ecc7-5513-4232-babe-143389a919c2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:47.651636 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1491ecc7-5513-4232-babe-143389a919c2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:47.652243 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1491ecc7-5513-4232-babe-143389a919c2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:47.652686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1491ecc7-5513-4232-babe-143389a919c2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:47.653423 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1491ecc7-5513-4232-babe-143389a919c2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:47.654655 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1491ecc7-5513-4232-babe-143389a919c2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:47.655548 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1491ecc7-5513-4232-babe-143389a919c2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:48.106605 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:48.107794 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] [instance: 03c3c04f-2850-480f-8d15-6f15327734e0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:48.114865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:48.115350 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:48.115785 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:48.116485 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:48.116931 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:48.117357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:48.134253 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] HTTP exception thrown: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 could not be found. Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Request method failure captured: Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 HTTP/1.1 Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ9DkOnCPmnFPAYepq2loImDxYdNr5cwNuK0FipLPZmtXhqlKuDFY5HfbRZ8DMteGiPseX8yjuTJWmYc2b3iCsNPJMR1z0rUaFXEopQsT81Rl7L5mAg-N_9p2t6IMAHa6c8ew6fROX3yi9quklri0gQb_DmLFnvmFFkO56rdzMKEeI4 Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 020d4019643f4499b5162717fbf7b5b5 Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-TestMinimumBasicScenario-1711789739 Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:48.135998 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/020d4019643f4499b5162717fbf7b5b5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_020d4019643f4499b5162717fbf7b5b5", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/020d4019643f4499b5162717fbf7b5b5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/020d4019643f4499b5162717fbf7b5b5"}]}] Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-TestMinimumBasicScenario-1711789739 Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 020d4019643f4499b5162717fbf7b5b5 Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1711789739 Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-TestMinimumBasicScenario-1711789739-project Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 14ddd15ab87a4d0fbea1d6dfa910f8ce Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-TestMinimumBasicScenario-1711789739-project Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 could not be found. Oct 20 09:07:48.137577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '03c3c04f-2850-480f-8d15-6f15327734e0'} Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 could not be found. Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:48.139057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 could not be found. Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 could not be found. Oct 20 09:07:48.140532 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:48.142461 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Returning 404 to user: Instance 03c3c04f-2850-480f-8d15-6f15327734e0 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:48.143900 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d9784a33-b982-4767-afcb-69d4740d7413 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0" status: 404 len: 111 microversion: 2.1 time: 0.042511 Oct 20 09:07:48.144468 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 513/1023] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:48 2022] GET /compute/v2.1/servers/03c3c04f-2850-480f-8d15-6f15327734e0 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:48.154882 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e1474031-4e28-46a1-9831-9a334ea19eb5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] Calling method '.version_select at 0x7f5afbac70d0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:48.178745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e1474031-4e28-46a1-9831-9a334ea19eb5 tempest-TestMinimumBasicScenario-1711789739 tempest-TestMinimumBasicScenario-1711789739-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1771110173" status: 202 len: 0 microversion: 2.1 time: 0.029466 Oct 20 09:07:48.179319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 514/1024] 10.222.0.36 () {58 vars in 1272 bytes} [Thu Oct 20 09:07:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1771110173 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:07:48.271471 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1491ecc7-5513-4232-babe-143389a919c2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:48.280031 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1491ecc7-5513-4232-babe-143389a919c2 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1674 microversion: 2.1 time: 0.644612 Oct 20 09:07:48.299859 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 511/1025] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:47 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1674 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:49.042385 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-741d4a4e-927b-4694-b9b8-e75acbed3232 req-8d4258ab-d2da-469f-8f2c-ba782cd378b4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "814ac79f-b35d-4aac-9b42-9f4ae8a83322", "name": "network-vif-deleted", "tag": "df10e6d7-6494-4543-851c-4aa8a550df87"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:07:49.057507 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-741d4a4e-927b-4694-b9b8-e75acbed3232 req-8d4258ab-d2da-469f-8f2c-ba782cd378b4 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:49.058383 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-741d4a4e-927b-4694-b9b8-e75acbed3232 req-8d4258ab-d2da-469f-8f2c-ba782cd378b4 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:49.072283 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-741d4a4e-927b-4694-b9b8-e75acbed3232 req-8d4258ab-d2da-469f-8f2c-ba782cd378b4 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:49.091822 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-741d4a4e-927b-4694-b9b8-e75acbed3232 req-8d4258ab-d2da-469f-8f2c-ba782cd378b4 service nova] Creating event network-vif-deleted:df10e6d7-6494-4543-851c-4aa8a550df87 for instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 on hv-cinder-maste Oct 20 09:07:49.097735 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-741d4a4e-927b-4694-b9b8-e75acbed3232 req-8d4258ab-d2da-469f-8f2c-ba782cd378b4 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063815 Oct 20 09:07:49.101517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 515/1026] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:07:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:07:49.306974 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:49.308147 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:49.318982 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:49.320158 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:49.320742 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:49.321634 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:49.322152 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:49.322625 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:49.368486 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:49.369013 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:49.369629 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:49.370241 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lazy-loading 'fault' on Instance uuid 814ac79f-b35d-4aac-9b42-9f4ae8a83322 {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:07:49.426639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'be1f82d7a0a84fbebdd0dffef9f098ea', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1061324903c42ed9f0306471a113b82', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:07:49.435559 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-959be979-cf6a-47ad-aa23-0d13e7730bf7 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 200 len: 1328 microversion: 2.1 time: 0.132501 Oct 20 09:07:49.436320 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 512/1027] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:49 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 1328 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:07:50.460744 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:50.462106 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:50.469619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:50.470391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:50.471006 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:50.472412 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:50.473635 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:50.474216 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:50.489845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] HTTP exception thrown: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Request method failure captured: Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 HTTP/1.1 Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ6P2OBFBjdaYYtKrV3A-9SXZZgUw8RpWuJ-V6FSIWbT0Iw-wZu5xI5JyTjvLIGVPVEPHBdKyfJ4iB2l_FB2zdybAqnizvM3OWPS2sy4Bn-3vwc1I1O0CXWMaGpV4UeI2CAf6XMPDX2yXehkwWHpa7ETUaM5XHs5f4WSxMNmwxhKIGk Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:50.491543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_f1061324903c42ed9f0306471a113b82", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/f1061324903c42ed9f0306471a113b82"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}] Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: be1f82d7a0a84fbebdd0dffef9f098ea Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:50.493013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '814ac79f-b35d-4aac-9b42-9f4ae8a83322'} Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:50.493782 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:50.495683 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:50.498886 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Returning 404 to user: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:50.498886 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-66278956-ba97-4c28-adc4-b26efc5943a4 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 404 len: 111 microversion: 2.1 time: 0.037192 Oct 20 09:07:50.503731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a8422e8b-e31a-4263-8d66-9388afaa5ad5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '.version_select at 0x7f5afbb0d790>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:50.506939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 516/1028] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:50 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:50.525531 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a8422e8b-e31a-4263-8d66-9388afaa5ad5 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2003739164" status: 202 len: 0 microversion: 2.1 time: 0.025848 Oct 20 09:07:50.526294 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 513/1029] 10.222.0.36 () {58 vars in 1221 bytes} [Thu Oct 20 09:07:50 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2003739164 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:07:59.239003 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:59.239512 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:59.245704 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:59.246187 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:59.246661 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:59.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:59.248338 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:59.248767 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:59.262087 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] HTTP exception thrown: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Request method failure captured: Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: DELETE /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 HTTP/1.1 Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ6P2OBFBjdaYYtKrV3A-9SXZZgUw8RpWuJ-V6FSIWbT0Iw-wZu5xI5JyTjvLIGVPVEPHBdKyfJ4iB2l_FB2zdybAqnizvM3OWPS2sy4Bn-3vwc1I1O0CXWMaGpV4UeI2CAf6XMPDX2yXehkwWHpa7ETUaM5XHs5f4WSxMNmwxhKIGk Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:07:59.264311 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_f1061324903c42ed9f0306471a113b82", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/f1061324903c42ed9f0306471a113b82"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}] Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: be1f82d7a0a84fbebdd0dffef9f098ea Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:59.265937 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '814ac79f-b35d-4aac-9b42-9f4ae8a83322'} Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:59.266828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:59.268290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:07:59.269799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Returning 404 to user: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:59.269799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-56a19911-245c-46f1-9940-b0abf02e7fdb tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "DELETE /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 404 len: 111 microversion: 2.1 time: 0.030957 Oct 20 09:07:59.269799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 517/1030] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:07:59 2022] DELETE /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:07:59.274986 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:07:59.275600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 814ac79f-b35d-4aac-9b42-9f4ae8a83322] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:07:59.282138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:59.282672 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:59.283160 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:59.283884 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:07:59.284340 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:07:59.284769 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:07:59.298693 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] HTTP exception thrown: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Request method failure captured: Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 HTTP/1.1 Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ6P2OBFBjdaYYtKrV3A-9SXZZgUw8RpWuJ-V6FSIWbT0Iw-wZu5xI5JyTjvLIGVPVEPHBdKyfJ4iB2l_FB2zdybAqnizvM3OWPS2sy4Bn-3vwc1I1O0CXWMaGpV4UeI2CAf6XMPDX2yXehkwWHpa7ETUaM5XHs5f4WSxMNmwxhKIGk Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:07:59.300210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_f1061324903c42ed9f0306471a113b82", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/f1061324903c42ed9f0306471a113b82"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}] Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: be1f82d7a0a84fbebdd0dffef9f098ea Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:59.301995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '814ac79f-b35d-4aac-9b42-9f4ae8a83322'} Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:07:59.303047 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. Oct 20 09:07:59.304476 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:07:59.305872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Returning 404 to user: Instance 814ac79f-b35d-4aac-9b42-9f4ae8a83322 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:07:59.305872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-d33fefd3-4d8c-4071-8a0e-3c9c195e434b tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322" status: 404 len: 111 microversion: 2.1 time: 0.031266 Oct 20 09:07:59.305872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 514/1031] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:07:59 2022] GET /compute/v2.1/servers/814ac79f-b35d-4aac-9b42-9f4ae8a83322 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:08:00.440491 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:00.440987 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:00.447400 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:00.447883 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:00.448346 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:00.449030 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:00.449484 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:00.449909 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:00.463084 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] HTTP exception thrown: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Request method failure captured: Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: DELETE /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 HTTP/1.1 Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ6P2OBFBjdaYYtKrV3A-9SXZZgUw8RpWuJ-V6FSIWbT0Iw-wZu5xI5JyTjvLIGVPVEPHBdKyfJ4iB2l_FB2zdybAqnizvM3OWPS2sy4Bn-3vwc1I1O0CXWMaGpV4UeI2CAf6XMPDX2yXehkwWHpa7ETUaM5XHs5f4WSxMNmwxhKIGk Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:08:00.464609 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_f1061324903c42ed9f0306471a113b82", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/f1061324903c42ed9f0306471a113b82"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}] Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: be1f82d7a0a84fbebdd0dffef9f098ea Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:08:00.466564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '26ae0069-be67-4921-aca3-4cfae5214cf5'} Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:08:00.467618 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:08:00.469108 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:00.470690 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Returning 404 to user: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:08:00.470690 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8ddc486a-7c9a-4a70-a8e3-eda3efa8711f tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "DELETE /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 404 len: 111 microversion: 2.1 time: 0.029756 Oct 20 09:08:00.470690 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 518/1032] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:08:00 2022] DELETE /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:08:00.474733 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:00.475392 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] [instance: 26ae0069-be67-4921-aca3-4cfae5214cf5] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:00.477404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=70,nova_api:DELETE=2,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:00.483186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:00.483666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:00.484103 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:00.484812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:00.485264 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:00.485692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:00.502400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] HTTP exception thrown: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Request method failure captured: Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 HTTP/1.1 Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ6P2OBFBjdaYYtKrV3A-9SXZZgUw8RpWuJ-V6FSIWbT0Iw-wZu5xI5JyTjvLIGVPVEPHBdKyfJ4iB2l_FB2zdybAqnizvM3OWPS2sy4Bn-3vwc1I1O0CXWMaGpV4UeI2CAf6XMPDX2yXehkwWHpa7ETUaM5XHs5f4WSxMNmwxhKIGk Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:08:00.504274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_f1061324903c42ed9f0306471a113b82", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/f1061324903c42ed9f0306471a113b82"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/f1061324903c42ed9f0306471a113b82"}]}] Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: f1061324903c42ed9f0306471a113b82 Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeTestJSON-726615277 Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: be1f82d7a0a84fbebdd0dffef9f098ea Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeTestJSON-726615277-project Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:08:00.505688 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '26ae0069-be67-4921-aca3-4cfae5214cf5'} Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:08:00.506447 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. Oct 20 09:08:00.507810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:00.508923 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] Returning 404 to user: Instance 26ae0069-be67-4921-aca3-4cfae5214cf5 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:08:00.508923 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3fe15706-b158-471f-b3b1-ac2da7486986 tempest-AttachVolumeTestJSON-726615277 tempest-AttachVolumeTestJSON-726615277-project] 10.222.0.36 "GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5" status: 404 len: 111 microversion: 2.1 time: 0.034830 Oct 20 09:08:00.508923 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 515/1033] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:00 2022] GET /compute/v2.1/servers/26ae0069-be67-4921-aca3-4cfae5214cf5 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:08:00.554798 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-879562910", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1651321010", "security_groups": [{"name": "tempest-secgroup-smoke-66517357"}], "networks": [{"uuid": "97e3a71a-7101-44af-a502-b2d032e40dce"}], "block_device_mapping_v2": [{"uuid": "cfd77c44-6939-4779-8585-30a4876c9ea6", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:08:00.695411 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] validate_networks() for [('97e3a71a-7101-44af-a502-b2d032e40dce', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:08:01.383980 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:08:01.384410 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:08:01.384881 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:08:01.385288 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:08:01.385697 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:08:01.397191 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:01.397677 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:01.398114 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:01.399013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:01.399468 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:01.399895 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:01.417052 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Getting quotas for project 45d1505e085847329f7480c9bc08f1d8. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:08:01.421130 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Getting quotas for user 21311bc5eef742f5a9eb81462a701ceb and project 45d1505e085847329f7480c9bc08f1d8. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:08:01.480778 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:08:01.483545 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='cfd77c44-6939-4779-8585-30a4876c9ea6',volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:08:01.615517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-05ea15fc-84c9-4086-82f6-27f83d36cd04 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 1.064605 Oct 20 09:08:01.616343 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 519/1034] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:08:00 2022] POST /compute/v2.1/servers => generated 404 bytes in 1066 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:08:01.627721 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-18fecbc3-8756-40dd-bf6f-c053329b580e tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:01.628322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-18fecbc3-8756-40dd-bf6f-c053329b580e tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:01.688011 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-18fecbc3-8756-40dd-bf6f-c053329b580e tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:01.695441 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-18fecbc3-8756-40dd-bf6f-c053329b580e tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1088 microversion: 2.1 time: 0.071315 Oct 20 09:08:01.696009 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 516/1035] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:01 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1088 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:02.720282 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-11ca405b-9f89-462b-bd96-5b51ca5ba8a7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:02.721583 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-11ca405b-9f89-462b-bd96-5b51ca5ba8a7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:02.729565 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11ca405b-9f89-462b-bd96-5b51ca5ba8a7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:02.730059 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11ca405b-9f89-462b-bd96-5b51ca5ba8a7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:02.730517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11ca405b-9f89-462b-bd96-5b51ca5ba8a7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:02.731753 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11ca405b-9f89-462b-bd96-5b51ca5ba8a7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:02.732415 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11ca405b-9f89-462b-bd96-5b51ca5ba8a7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:02.732995 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-11ca405b-9f89-462b-bd96-5b51ca5ba8a7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:02.805707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-11ca405b-9f89-462b-bd96-5b51ca5ba8a7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:02.819865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-11ca405b-9f89-462b-bd96-5b51ca5ba8a7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1138 microversion: 2.1 time: 0.104864 Oct 20 09:08:02.822510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 520/1036] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:02 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1138 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:03.843713 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-1ee1c3c7-e65f-4bdd-8c52-16669c55ed6a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:03.844478 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-1ee1c3c7-e65f-4bdd-8c52-16669c55ed6a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:03.856059 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ee1c3c7-e65f-4bdd-8c52-16669c55ed6a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:03.857471 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ee1c3c7-e65f-4bdd-8c52-16669c55ed6a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:03.858105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ee1c3c7-e65f-4bdd-8c52-16669c55ed6a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:03.858928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ee1c3c7-e65f-4bdd-8c52-16669c55ed6a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:03.859472 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ee1c3c7-e65f-4bdd-8c52-16669c55ed6a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:03.860166 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-1ee1c3c7-e65f-4bdd-8c52-16669c55ed6a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:03.935937 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-1ee1c3c7-e65f-4bdd-8c52-16669c55ed6a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:03.943780 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-1ee1c3c7-e65f-4bdd-8c52-16669c55ed6a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1186 microversion: 2.1 time: 0.104224 Oct 20 09:08:03.944352 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 517/1037] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:03 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1186 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:08:04.972251 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-51bff1bb-0cae-4183-bf41-a80a78420b1f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:04.973004 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-51bff1bb-0cae-4183-bf41-a80a78420b1f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:04.982557 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-51bff1bb-0cae-4183-bf41-a80a78420b1f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:04.983183 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-51bff1bb-0cae-4183-bf41-a80a78420b1f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:04.983850 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-51bff1bb-0cae-4183-bf41-a80a78420b1f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:04.984761 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-51bff1bb-0cae-4183-bf41-a80a78420b1f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:04.985368 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-51bff1bb-0cae-4183-bf41-a80a78420b1f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:04.986107 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-51bff1bb-0cae-4183-bf41-a80a78420b1f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:05.071393 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-51bff1bb-0cae-4183-bf41-a80a78420b1f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:05.079357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-51bff1bb-0cae-4183-bf41-a80a78420b1f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1204 microversion: 2.1 time: 0.115042 Oct 20 09:08:05.080242 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 521/1038] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:04 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1204 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:06.088249 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-e69d4b69-52df-4893-9677-29692f57e9e7 req-cfd75800-15d8-4b88-8a7d-4b4c43bdf16a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c8a04d20-065b-47fd-b7e2-8238e34be187", "tag": "731aa0eb-33e7-4deb-b323-29e7908786e9"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:08:06.103142 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f2692087-1dc3-45a0-b113-63f013346949 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:06.103755 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f2692087-1dc3-45a0-b113-63f013346949 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:06.112452 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-e69d4b69-52df-4893-9677-29692f57e9e7 req-cfd75800-15d8-4b88-8a7d-4b4c43bdf16a service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:06.112947 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-e69d4b69-52df-4893-9677-29692f57e9e7 req-cfd75800-15d8-4b88-8a7d-4b4c43bdf16a service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:06.113381 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-e69d4b69-52df-4893-9677-29692f57e9e7 req-cfd75800-15d8-4b88-8a7d-4b4c43bdf16a service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:06.115381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f2692087-1dc3-45a0-b113-63f013346949 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:06.119180 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f2692087-1dc3-45a0-b113-63f013346949 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:06.119713 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f2692087-1dc3-45a0-b113-63f013346949 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:06.120511 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f2692087-1dc3-45a0-b113-63f013346949 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:06.120974 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f2692087-1dc3-45a0-b113-63f013346949 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:06.121400 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f2692087-1dc3-45a0-b113-63f013346949 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:06.134513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-e69d4b69-52df-4893-9677-29692f57e9e7 req-cfd75800-15d8-4b88-8a7d-4b4c43bdf16a service nova] Creating event network-changed:731aa0eb-33e7-4deb-b323-29e7908786e9 for instance c8a04d20-065b-47fd-b7e2-8238e34be187 on hv-cinder-maste Oct 20 09:08:06.143843 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-e69d4b69-52df-4893-9677-29692f57e9e7 req-cfd75800-15d8-4b88-8a7d-4b4c43bdf16a service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059662 Oct 20 09:08:06.144712 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 518/1039] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:08:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:08:06.269984 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f2692087-1dc3-45a0-b113-63f013346949 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:06.277827 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f2692087-1dc3-45a0-b113-63f013346949 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.178338 Oct 20 09:08:06.278547 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 522/1040] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:06 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:08:07.301074 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b4cf4639-68c5-4410-b009-86bf4c7a8da3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:07.301897 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b4cf4639-68c5-4410-b009-86bf4c7a8da3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:07.308466 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4cf4639-68c5-4410-b009-86bf4c7a8da3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:07.309122 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4cf4639-68c5-4410-b009-86bf4c7a8da3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:07.309645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4cf4639-68c5-4410-b009-86bf4c7a8da3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:07.310352 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4cf4639-68c5-4410-b009-86bf4c7a8da3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:07.310823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4cf4639-68c5-4410-b009-86bf4c7a8da3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:07.311254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b4cf4639-68c5-4410-b009-86bf4c7a8da3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:07.448983 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b4cf4639-68c5-4410-b009-86bf4c7a8da3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:07.458297 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b4cf4639-68c5-4410-b009-86bf4c7a8da3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.161324 Oct 20 09:08:07.459462 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 519/1041] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:07 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:08.482500 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-18d154a1-09d9-469a-8ae3-96175831e6d2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:08.483122 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-18d154a1-09d9-469a-8ae3-96175831e6d2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:08.489952 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-18d154a1-09d9-469a-8ae3-96175831e6d2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:08.490543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-18d154a1-09d9-469a-8ae3-96175831e6d2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:08.490982 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-18d154a1-09d9-469a-8ae3-96175831e6d2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:08.491702 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-18d154a1-09d9-469a-8ae3-96175831e6d2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:08.492148 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-18d154a1-09d9-469a-8ae3-96175831e6d2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:08.492635 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-18d154a1-09d9-469a-8ae3-96175831e6d2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:08.627828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-18d154a1-09d9-469a-8ae3-96175831e6d2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:08.635695 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-18d154a1-09d9-469a-8ae3-96175831e6d2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.157618 Oct 20 09:08:08.636344 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 523/1042] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:08 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:09.659096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-30924059-7879-4aa8-93ca-d3617052aacd tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:09.659644 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-30924059-7879-4aa8-93ca-d3617052aacd tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:09.667899 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30924059-7879-4aa8-93ca-d3617052aacd tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:09.668424 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30924059-7879-4aa8-93ca-d3617052aacd tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:09.669018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30924059-7879-4aa8-93ca-d3617052aacd tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:09.669726 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30924059-7879-4aa8-93ca-d3617052aacd tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:09.670547 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30924059-7879-4aa8-93ca-d3617052aacd tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:09.671795 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30924059-7879-4aa8-93ca-d3617052aacd tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:09.885066 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-30924059-7879-4aa8-93ca-d3617052aacd tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:09.893618 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-30924059-7879-4aa8-93ca-d3617052aacd tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.238433 Oct 20 09:08:09.894185 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 520/1043] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:09 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:10.916889 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bd195280-6999-4561-aa51-7590d563fb4c tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:10.917524 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bd195280-6999-4561-aa51-7590d563fb4c tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:10.923873 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd195280-6999-4561-aa51-7590d563fb4c tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:10.924360 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd195280-6999-4561-aa51-7590d563fb4c tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:10.924791 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd195280-6999-4561-aa51-7590d563fb4c tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:10.925482 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd195280-6999-4561-aa51-7590d563fb4c tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:10.925934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd195280-6999-4561-aa51-7590d563fb4c tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:10.926358 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bd195280-6999-4561-aa51-7590d563fb4c tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:11.061715 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bd195280-6999-4561-aa51-7590d563fb4c tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:11.069550 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bd195280-6999-4561-aa51-7590d563fb4c tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.156548 Oct 20 09:08:11.070099 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 524/1044] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:10 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:12.092903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-be4e26d6-bd00-4a03-a1c0-84b1a73b3cfe tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:12.093527 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-be4e26d6-bd00-4a03-a1c0-84b1a73b3cfe tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:12.095961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:12.101446 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be4e26d6-bd00-4a03-a1c0-84b1a73b3cfe tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:12.101939 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be4e26d6-bd00-4a03-a1c0-84b1a73b3cfe tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:12.102382 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be4e26d6-bd00-4a03-a1c0-84b1a73b3cfe tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:12.103104 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be4e26d6-bd00-4a03-a1c0-84b1a73b3cfe tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:12.103567 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be4e26d6-bd00-4a03-a1c0-84b1a73b3cfe tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:12.103985 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-be4e26d6-bd00-4a03-a1c0-84b1a73b3cfe tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:12.107419 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:12.244278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-be4e26d6-bd00-4a03-a1c0-84b1a73b3cfe tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:12.252748 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-be4e26d6-bd00-4a03-a1c0-84b1a73b3cfe tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.164515 Oct 20 09:08:12.253850 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 521/1045] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:12 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:13.276485 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a1dca07e-4514-484b-a5a8-0254fa0bd96d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:13.277175 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a1dca07e-4514-484b-a5a8-0254fa0bd96d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:13.279721 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:13.280651 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=6 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:13.281984 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=61,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:13.288918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a1dca07e-4514-484b-a5a8-0254fa0bd96d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:13.289403 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a1dca07e-4514-484b-a5a8-0254fa0bd96d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:13.289835 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a1dca07e-4514-484b-a5a8-0254fa0bd96d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:13.290547 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a1dca07e-4514-484b-a5a8-0254fa0bd96d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:13.291000 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a1dca07e-4514-484b-a5a8-0254fa0bd96d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:13.291423 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a1dca07e-4514-484b-a5a8-0254fa0bd96d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:13.295719 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:13.431805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a1dca07e-4514-484b-a5a8-0254fa0bd96d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:13.440306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a1dca07e-4514-484b-a5a8-0254fa0bd96d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.167648 Oct 20 09:08:13.440953 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 525/1046] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:13 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:14.462822 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a72c1b74-6399-4179-865d-31a1a750b601 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:14.463432 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a72c1b74-6399-4179-865d-31a1a750b601 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:14.470780 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a72c1b74-6399-4179-865d-31a1a750b601 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:14.471187 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a72c1b74-6399-4179-865d-31a1a750b601 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:14.471620 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a72c1b74-6399-4179-865d-31a1a750b601 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:14.472475 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a72c1b74-6399-4179-865d-31a1a750b601 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:14.472928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a72c1b74-6399-4179-865d-31a1a750b601 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:14.473351 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a72c1b74-6399-4179-865d-31a1a750b601 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:14.611124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a72c1b74-6399-4179-865d-31a1a750b601 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:14.618976 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a72c1b74-6399-4179-865d-31a1a750b601 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.160143 Oct 20 09:08:14.619541 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 522/1047] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:14 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:15.420452 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6255345a-139f-473a-9dab-7044dcb9fc18 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:15.422977 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6255345a-139f-473a-9dab-7044dcb9fc18 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:15.430370 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6255345a-139f-473a-9dab-7044dcb9fc18 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:15.430884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6255345a-139f-473a-9dab-7044dcb9fc18 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:15.431317 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6255345a-139f-473a-9dab-7044dcb9fc18 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:15.432005 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6255345a-139f-473a-9dab-7044dcb9fc18 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:15.432458 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6255345a-139f-473a-9dab-7044dcb9fc18 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:15.432882 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6255345a-139f-473a-9dab-7044dcb9fc18 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:15.463586 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6255345a-139f-473a-9dab-7044dcb9fc18 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.048196 Oct 20 09:08:15.464154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 526/1048] 10.222.0.36 () {60 vars in 1343 bytes} [Thu Oct 20 09:08:15 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:15.535941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5f8a8f5e-c199-4da5-a4a6-20356a05cae1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:15.536600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5f8a8f5e-c199-4da5-a4a6-20356a05cae1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:15.543262 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5f8a8f5e-c199-4da5-a4a6-20356a05cae1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:15.544271 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5f8a8f5e-c199-4da5-a4a6-20356a05cae1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:15.544712 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5f8a8f5e-c199-4da5-a4a6-20356a05cae1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:15.545412 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5f8a8f5e-c199-4da5-a4a6-20356a05cae1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:15.545868 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5f8a8f5e-c199-4da5-a4a6-20356a05cae1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:15.546300 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5f8a8f5e-c199-4da5-a4a6-20356a05cae1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:15.577599 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5f8a8f5e-c199-4da5-a4a6-20356a05cae1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:08:15.645412 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-33c8acd3-4f4e-41fe-865b-b14dd165c8df tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:15.645764 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-33c8acd3-4f4e-41fe-865b-b14dd165c8df tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:15.651769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33c8acd3-4f4e-41fe-865b-b14dd165c8df tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:15.653224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33c8acd3-4f4e-41fe-865b-b14dd165c8df tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:15.654175 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33c8acd3-4f4e-41fe-865b-b14dd165c8df tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:15.655105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33c8acd3-4f4e-41fe-865b-b14dd165c8df tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:15.655717 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33c8acd3-4f4e-41fe-865b-b14dd165c8df tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:15.656530 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-33c8acd3-4f4e-41fe-865b-b14dd165c8df tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:15.661744 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5f8a8f5e-c199-4da5-a4a6-20356a05cae1 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "DELETE /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 204 len: 0 microversion: 2.20 time: 0.129071 Oct 20 09:08:15.662955 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 523/1049] 10.222.0.36 () {60 vars in 1280 bytes} [Thu Oct 20 09:08:15 2022] DELETE /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 20 09:08:15.674873 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:15.675925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:15.682757 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:15.683331 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:15.684410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:15.685334 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:15.686053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:15.686778 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:15.807088 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-33c8acd3-4f4e-41fe-865b-b14dd165c8df tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:15.815853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-33c8acd3-4f4e-41fe-865b-b14dd165c8df tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.174654 Oct 20 09:08:15.816634 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 527/1050] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:15 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:15.844011 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:15.852096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:15.853536 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:15.854898 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-632e2cfc-e614-4dee-912a-f9073a5c6679 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.186995 Oct 20 09:08:15.855455 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 524/1051] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:08:15 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:08:16.844384 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-62c8d3b4-e4e8-4f2b-b439-2d82a13b7b8d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:16.845004 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-62c8d3b4-e4e8-4f2b-b439-2d82a13b7b8d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:16.851425 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62c8d3b4-e4e8-4f2b-b439-2d82a13b7b8d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:16.851904 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62c8d3b4-e4e8-4f2b-b439-2d82a13b7b8d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:16.852338 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62c8d3b4-e4e8-4f2b-b439-2d82a13b7b8d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:16.853100 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62c8d3b4-e4e8-4f2b-b439-2d82a13b7b8d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:16.853597 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62c8d3b4-e4e8-4f2b-b439-2d82a13b7b8d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:16.854015 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-62c8d3b4-e4e8-4f2b-b439-2d82a13b7b8d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:16.880653 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:16.881913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:16.889057 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:16.889473 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:16.889909 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:16.890641 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:16.891133 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:16.891507 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:17.007334 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-62c8d3b4-e4e8-4f2b-b439-2d82a13b7b8d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:17.015851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-62c8d3b4-e4e8-4f2b-b439-2d82a13b7b8d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.175496 Oct 20 09:08:17.016421 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 528/1052] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:16 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:17.037152 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:17.045674 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:17.047162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:17.048330 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e8302c86-0f02-4f4a-b494-f8413d476105 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.171566 Oct 20 09:08:17.048884 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 525/1053] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:08:16 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 20 09:08:18.038219 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0c367cdd-fe7c-409c-be64-9d7aa224f95f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:18.038853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0c367cdd-fe7c-409c-be64-9d7aa224f95f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:18.045081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c367cdd-fe7c-409c-be64-9d7aa224f95f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:18.045573 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c367cdd-fe7c-409c-be64-9d7aa224f95f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:18.046015 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c367cdd-fe7c-409c-be64-9d7aa224f95f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:18.046742 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c367cdd-fe7c-409c-be64-9d7aa224f95f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:18.047385 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c367cdd-fe7c-409c-be64-9d7aa224f95f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:18.047981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0c367cdd-fe7c-409c-be64-9d7aa224f95f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:18.078144 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:18.078853 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:18.086160 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:18.086676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:18.093209 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:18.094201 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:18.094858 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:18.095456 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:18.248329 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0c367cdd-fe7c-409c-be64-9d7aa224f95f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:18.256262 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0c367cdd-fe7c-409c-be64-9d7aa224f95f tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.221803 Oct 20 09:08:18.257022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 529/1054] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:18 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:18.327503 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:18.335473 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:18.336935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:18.338082 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b699ba7f-22f7-48e0-98c4-914535d5737c tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1725 microversion: 2.20 time: 0.266751 Oct 20 09:08:18.338654 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 526/1055] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:08:18 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1725 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:08:18.644820 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-f5174b5e-f1db-4670-b0a3-1eecd68e28d8 req-ce739a3d-09b4-484f-86ec-7669c0318978 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7facb789-628a-44aa-b0ec-7f6aee5617dd", "name": "network-vif-deleted", "tag": "f17302af-3ee4-46d2-a6fe-a973b183f4e4"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:08:18.657161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-f5174b5e-f1db-4670-b0a3-1eecd68e28d8 req-ce739a3d-09b4-484f-86ec-7669c0318978 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:18.657637 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-f5174b5e-f1db-4670-b0a3-1eecd68e28d8 req-ce739a3d-09b4-484f-86ec-7669c0318978 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:18.658057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-f5174b5e-f1db-4670-b0a3-1eecd68e28d8 req-ce739a3d-09b4-484f-86ec-7669c0318978 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:18.670098 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-f5174b5e-f1db-4670-b0a3-1eecd68e28d8 req-ce739a3d-09b4-484f-86ec-7669c0318978 service nova] Creating event network-vif-deleted:f17302af-3ee4-46d2-a6fe-a973b183f4e4 for instance 7facb789-628a-44aa-b0ec-7f6aee5617dd on hv-cinder-maste Oct 20 09:08:18.675751 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-f5174b5e-f1db-4670-b0a3-1eecd68e28d8 req-ce739a3d-09b4-484f-86ec-7669c0318978 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035458 Oct 20 09:08:18.676757 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 530/1056] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:08:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:08:19.279704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bce43953-0952-47ab-b96c-9ffa5b32136d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:19.280304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bce43953-0952-47ab-b96c-9ffa5b32136d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:19.295285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bce43953-0952-47ab-b96c-9ffa5b32136d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:19.296026 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bce43953-0952-47ab-b96c-9ffa5b32136d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:19.296657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bce43953-0952-47ab-b96c-9ffa5b32136d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:19.297524 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bce43953-0952-47ab-b96c-9ffa5b32136d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:19.298130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bce43953-0952-47ab-b96c-9ffa5b32136d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:19.298745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bce43953-0952-47ab-b96c-9ffa5b32136d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:19.364290 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:19.365582 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:19.376503 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:19.378353 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:19.379153 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:19.380934 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:19.382169 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:19.382805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:19.430465 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:19.432656 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:19.433146 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:19.433699 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lazy-loading 'fault' on Instance uuid 7facb789-628a-44aa-b0ec-7f6aee5617dd {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:08:19.471644 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bce43953-0952-47ab-b96c-9ffa5b32136d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:19.489765 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:19.490292 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bce43953-0952-47ab-b96c-9ffa5b32136d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.214360 Oct 20 09:08:19.506747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 527/1057] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:19 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:19.511501 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:19.513626 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fd9733d1bb2b4520b70090542615ddf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '869b5cf3a9264fd3b70ae656d7814cfb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:19.530543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0274925a-9bcc-4e7b-bf52-753f75abf547 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 200 len: 1372 microversion: 2.20 time: 0.155516 Oct 20 09:08:19.535111 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 531/1058] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:08:19 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 1372 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 20 09:08:19.725052 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-47fff37e-7604-4a9b-a02e-96c3decf5697 req-8d08757e-c3eb-47f9-a14e-371a6f092ab5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c8a04d20-065b-47fd-b7e2-8238e34be187", "name": "network-vif-plugged", "status": "completed", "tag": "731aa0eb-33e7-4deb-b323-29e7908786e9"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:08:19.737933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-47fff37e-7604-4a9b-a02e-96c3decf5697 req-8d08757e-c3eb-47f9-a14e-371a6f092ab5 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:19.738747 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-47fff37e-7604-4a9b-a02e-96c3decf5697 req-8d08757e-c3eb-47f9-a14e-371a6f092ab5 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:19.739415 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-47fff37e-7604-4a9b-a02e-96c3decf5697 req-8d08757e-c3eb-47f9-a14e-371a6f092ab5 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:19.759900 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-47fff37e-7604-4a9b-a02e-96c3decf5697 req-8d08757e-c3eb-47f9-a14e-371a6f092ab5 service nova] Creating event network-vif-plugged:731aa0eb-33e7-4deb-b323-29e7908786e9 for instance c8a04d20-065b-47fd-b7e2-8238e34be187 on hv-cinder-maste Oct 20 09:08:19.767764 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-47fff37e-7604-4a9b-a02e-96c3decf5697 req-8d08757e-c3eb-47f9-a14e-371a6f092ab5 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046496 Oct 20 09:08:19.769247 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 528/1059] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:08:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:08:20.531293 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1d993b82-ef05-425d-8034-211dae5b7baa tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:20.531907 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1d993b82-ef05-425d-8034-211dae5b7baa tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:20.538213 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d993b82-ef05-425d-8034-211dae5b7baa tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:20.538727 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d993b82-ef05-425d-8034-211dae5b7baa tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:20.539163 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d993b82-ef05-425d-8034-211dae5b7baa tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:20.539851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d993b82-ef05-425d-8034-211dae5b7baa tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:20.540304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d993b82-ef05-425d-8034-211dae5b7baa tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:20.540726 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1d993b82-ef05-425d-8034-211dae5b7baa tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:20.559965 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:20.560876 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:20.567053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:20.567539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:20.567979 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:20.568675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:20.569137 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:20.569559 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:20.586510 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] HTTP exception thrown: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Request method failure captured: Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd HTTP/1.1 Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ45rraxKtmAczlSJFs0dTqdOsnHgw3sd7cCCv7-jWXet363TlKZqjHREJ1szDoP0_9r30G6izG2SMxnb9AdlZuYz2zkipPGMfoApkn6nORm837Fs5_N4Zn-_5QRzx55zhX2116SyS1aHgjJH8Dc90kajhoUZL99KLxKxA6o6SKuG8g Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Openstack-Nova-Api-Version: 2.20 Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:08:20.587935 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_869b5cf3a9264fd3b70ae656d7814cfb", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}] Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: fd9733d1bb2b4520b70090542615ddf9 Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:20.590841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '7facb789-628a-44aa-b0ec-7f6aee5617dd'} Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:08:20.595589 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:20.596941 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:20.598085 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Returning 404 to user: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:08:20.598085 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-9ef656f7-7a8f-4383-b251-78e3c5f45127 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 404 len: 111 microversion: 2.20 time: 0.034340 Oct 20 09:08:20.598085 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 529/1060] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:08:20 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 20 09:08:20.599297 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-0b4425db-a707-46e2-9051-9069acbd9920 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '.version_select at 0x7f5afba9a4c0>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:20.619775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-0b4425db-a707-46e2-9051-9069acbd9920 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1573757010" status: 204 len: 0 microversion: 2.20 time: 0.023986 Oct 20 09:08:20.621116 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 530/1061] 10.222.0.36 () {60 vars in 1262 bytes} [Thu Oct 20 09:08:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1573757010 => generated 0 bytes in 26 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 20 09:08:20.732771 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1d993b82-ef05-425d-8034-211dae5b7baa tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:20.742518 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1d993b82-ef05-425d-8034-211dae5b7baa tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1258 microversion: 2.1 time: 0.215459 Oct 20 09:08:20.743451 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 532/1062] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:20 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1258 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:21.767926 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6e41892a-2ca6-4c16-a83f-f82a86bef548 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:21.769078 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6e41892a-2ca6-4c16-a83f-f82a86bef548 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:21.780448 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6e41892a-2ca6-4c16-a83f-f82a86bef548 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:21.781232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6e41892a-2ca6-4c16-a83f-f82a86bef548 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:21.782910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6e41892a-2ca6-4c16-a83f-f82a86bef548 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:21.784398 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6e41892a-2ca6-4c16-a83f-f82a86bef548 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:21.784966 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6e41892a-2ca6-4c16-a83f-f82a86bef548 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:21.785544 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6e41892a-2ca6-4c16-a83f-f82a86bef548 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:21.956222 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6e41892a-2ca6-4c16-a83f-f82a86bef548 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:21.963886 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6e41892a-2ca6-4c16-a83f-f82a86bef548 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1441 microversion: 2.1 time: 0.200324 Oct 20 09:08:21.964702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 531/1063] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:21 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1441 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:21.989273 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7738a48b-9220-4f24-8ac9-b6c98612fac7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:21.990319 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7738a48b-9220-4f24-8ac9-b6c98612fac7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:21.997143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7738a48b-9220-4f24-8ac9-b6c98612fac7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:21.997560 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7738a48b-9220-4f24-8ac9-b6c98612fac7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:21.997995 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7738a48b-9220-4f24-8ac9-b6c98612fac7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:21.998719 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7738a48b-9220-4f24-8ac9-b6c98612fac7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:21.999393 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7738a48b-9220-4f24-8ac9-b6c98612fac7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:22.000019 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7738a48b-9220-4f24-8ac9-b6c98612fac7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:22.149799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7738a48b-9220-4f24-8ac9-b6c98612fac7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:08:22.159471 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7738a48b-9220-4f24-8ac9-b6c98612fac7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1441 microversion: 2.1 time: 0.175287 Oct 20 09:08:22.161068 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 533/1064] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:08:21 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1441 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:22.439121 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:22.440117 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:22.447106 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:22.447903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:22.448548 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:22.450228 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:22.451741 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:22.452406 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:22.469908 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] HTTP exception thrown: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Request method failure captured: Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: DELETE /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd HTTP/1.1 Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ45rraxKtmAczlSJFs0dTqdOsnHgw3sd7cCCv7-jWXet363TlKZqjHREJ1szDoP0_9r30G6izG2SMxnb9AdlZuYz2zkipPGMfoApkn6nORm837Fs5_N4Zn-_5QRzx55zhX2116SyS1aHgjJH8Dc90kajhoUZL99KLxKxA6o6SKuG8g Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Openstack-Nova-Api-Version: 2.20 Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:08:22.471827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_869b5cf3a9264fd3b70ae656d7814cfb", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}] Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: fd9733d1bb2b4520b70090542615ddf9 Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:22.473261 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '7facb789-628a-44aa-b0ec-7f6aee5617dd'} Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:08:22.474019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:22.475388 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:22.476948 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Returning 404 to user: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:08:22.476948 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-33bf35da-31d1-483d-9ef6-b8da36f631c7 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "DELETE /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 404 len: 111 microversion: 2.20 time: 0.038437 Oct 20 09:08:22.476948 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 532/1065] 10.222.0.36 () {60 vars in 1280 bytes} [Thu Oct 20 09:08:22 2022] DELETE /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 20 09:08:22.483981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:22.484760 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: 7facb789-628a-44aa-b0ec-7f6aee5617dd] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:22.500380 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:22.501328 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:22.501803 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:22.503043 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:22.503548 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:22.503986 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:22.523831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] HTTP exception thrown: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Request method failure captured: Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd HTTP/1.1 Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ45rraxKtmAczlSJFs0dTqdOsnHgw3sd7cCCv7-jWXet363TlKZqjHREJ1szDoP0_9r30G6izG2SMxnb9AdlZuYz2zkipPGMfoApkn6nORm837Fs5_N4Zn-_5QRzx55zhX2116SyS1aHgjJH8Dc90kajhoUZL99KLxKxA6o6SKuG8g Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Openstack-Nova-Api-Version: 2.20 Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:08:22.525502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_869b5cf3a9264fd3b70ae656d7814cfb", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}] Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: fd9733d1bb2b4520b70090542615ddf9 Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:22.526961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '7facb789-628a-44aa-b0ec-7f6aee5617dd'} Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:08:22.527880 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. Oct 20 09:08:22.529657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:22.533604 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Returning 404 to user: Instance 7facb789-628a-44aa-b0ec-7f6aee5617dd could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:08:22.533604 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-73d9d897-1cce-45ca-9dc6-1724c9b9ebbb tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd" status: 404 len: 111 microversion: 2.20 time: 0.047897 Oct 20 09:08:22.533604 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 534/1066] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:08:22 2022] GET /compute/v2.1/servers/7facb789-628a-44aa-b0ec-7f6aee5617dd => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 20 09:08:23.684276 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:23.684819 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:23.691486 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:23.691971 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:23.692405 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:23.693096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:23.693573 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:23.694001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:23.707746 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] HTTP exception thrown: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Request method failure captured: Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: DELETE /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 HTTP/1.1 Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ45rraxKtmAczlSJFs0dTqdOsnHgw3sd7cCCv7-jWXet363TlKZqjHREJ1szDoP0_9r30G6izG2SMxnb9AdlZuYz2zkipPGMfoApkn6nORm837Fs5_N4Zn-_5QRzx55zhX2116SyS1aHgjJH8Dc90kajhoUZL99KLxKxA6o6SKuG8g Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Openstack-Nova-Api-Version: 2.20 Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:08:23.709356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_869b5cf3a9264fd3b70ae656d7814cfb", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}] Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: fd9733d1bb2b4520b70090542615ddf9 Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:08:23.711162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': 'f8d61907-24ca-4e74-917e-d218d0f89557'} Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:08:23.712034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:08:23.713686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:08:23.717401 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Returning 404 to user: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:08:23.717401 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6f85812e-1926-4a73-b8ed-d3251695c806 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "DELETE /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 404 len: 111 microversion: 2.20 time: 0.031001 Oct 20 09:08:23.717401 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 533/1067] 10.222.0.36 () {60 vars in 1280 bytes} [Thu Oct 20 09:08:23 2022] DELETE /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 20 09:08:23.719491 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:23.720099 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] [instance: f8d61907-24ca-4e74-917e-d218d0f89557] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:08:23.726802 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:23.727411 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:23.727712 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:23.728401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:23.728858 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:23.729275 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:23.745652 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] HTTP exception thrown: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Request method failure captured: Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 HTTP/1.1 Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ45rraxKtmAczlSJFs0dTqdOsnHgw3sd7cCCv7-jWXet363TlKZqjHREJ1szDoP0_9r30G6izG2SMxnb9AdlZuYz2zkipPGMfoApkn6nORm837Fs5_N4Zn-_5QRzx55zhX2116SyS1aHgjJH8Dc90kajhoUZL99KLxKxA6o6SKuG8g Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Openstack-Nova-Api-Version: 2.20 Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:08:23.747783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_869b5cf3a9264fd3b70ae656d7814cfb", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/869b5cf3a9264fd3b70ae656d7814cfb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/869b5cf3a9264fd3b70ae656d7814cfb"}]}] Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 869b5cf3a9264fd3b70ae656d7814cfb Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-344536273 Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: fd9733d1bb2b4520b70090542615ddf9 Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-AttachVolumeShelveTestJSON-344536273-project Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:08:23.752322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': 'f8d61907-24ca-4e74-917e-d218d0f89557'} Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:08:23.754298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. Oct 20 09:08:23.755916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:08:23.757174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] Returning 404 to user: Instance f8d61907-24ca-4e74-917e-d218d0f89557 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:08:23.757174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2209dd49-b9c6-4b26-ac21-6cf342afbd93 tempest-AttachVolumeShelveTestJSON-344536273 tempest-AttachVolumeShelveTestJSON-344536273-project] 10.222.0.36 "GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557" status: 404 len: 111 microversion: 2.20 time: 0.033511 Oct 20 09:08:23.757174 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 535/1068] 10.222.0.36 () {60 vars in 1277 bytes} [Thu Oct 20 09:08:23 2022] GET /compute/v2.1/servers/f8d61907-24ca-4e74-917e-d218d0f89557 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 20 09:08:24.532158 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-29c139c6-e3a2-466d-886e-750a22f93d33 req-14411075-5393-4e79-b7c3-1a0d50037927 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c8a04d20-065b-47fd-b7e2-8238e34be187", "tag": "731aa0eb-33e7-4deb-b323-29e7908786e9"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:08:24.541887 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-29c139c6-e3a2-466d-886e-750a22f93d33 req-14411075-5393-4e79-b7c3-1a0d50037927 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:08:24.542640 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-29c139c6-e3a2-466d-886e-750a22f93d33 req-14411075-5393-4e79-b7c3-1a0d50037927 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:08:24.543371 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-29c139c6-e3a2-466d-886e-750a22f93d33 req-14411075-5393-4e79-b7c3-1a0d50037927 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:08:24.555180 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-29c139c6-e3a2-466d-886e-750a22f93d33 req-14411075-5393-4e79-b7c3-1a0d50037927 service nova] Creating event network-changed:731aa0eb-33e7-4deb-b323-29e7908786e9 for instance c8a04d20-065b-47fd-b7e2-8238e34be187 on hv-cinder-maste Oct 20 09:08:24.562019 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-29c139c6-e3a2-466d-886e-750a22f93d33 req-14411075-5393-4e79-b7c3-1a0d50037927 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033523 Oct 20 09:08:24.563004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 534/1069] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:08:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:08:31.045792 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bd94168d-82e1-41af-8692-334a33ea8561 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afbb28b80>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:31.094340 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bd94168d-82e1-41af-8692-334a33ea8561 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/71e9c3c2-482e-43c1-a777-db79660314f6" status: 200 len: 351 microversion: 2.1 time: 0.052352 Oct 20 09:08:31.095073 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 536/1070] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:08:31 2022] GET /compute/v2.1/os-volumes/71e9c3c2-482e-43c1-a777-db79660314f6 => generated 351 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:08:32.453434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4b0c75a8-040e-4ab6-b624-fb1a2226bb2a tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afb9745e0>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:32.508461 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4b0c75a8-040e-4ab6-b624-fb1a2226bb2a tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/bfff4482-6692-4c3a-9f22-e1ec554ce7c8" status: 200 len: 350 microversion: 2.1 time: 0.060640 Oct 20 09:08:32.509053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 535/1071] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:08:32 2022] GET /compute/v2.1/os-volumes/bfff4482-6692-4c3a-9f22-e1ec554ce7c8 => generated 350 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:08:33.851945 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=23 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:33.852754 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=55 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:33.858044 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a6219def-a9a4-4925-9aa7-390b4b9371d6 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afbc53310>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:33.909010 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a6219def-a9a4-4925-9aa7-390b4b9371d6 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/cb2437c4-7c83-4232-a2c9-00583d79f9ba" status: 200 len: 350 microversion: 2.1 time: 0.058455 Oct 20 09:08:33.909663 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 537/1072] 10.222.0.36 () {58 vars in 1245 bytes} [Thu Oct 20 09:08:33 2022] GET /compute/v2.1/os-volumes/cb2437c4-7c83-4232-a2c9-00583d79f9ba => generated 350 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:08:33.924482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-cf0ee726-49b4-49b8-bbd0-de64590a529d tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afb9e44c0>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:34.009060 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-cf0ee726-49b4-49b8-bbd0-de64590a529d tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.088716 Oct 20 09:08:34.009794 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 536/1073] 10.222.0.36 () {58 vars in 1134 bytes} [Thu Oct 20 09:08:33 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:34.024273 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ba5b7f37-4081-45e2-b8d6-6153e922d05f tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afbacdc10>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:34.111224 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ba5b7f37-4081-45e2-b8d6-6153e922d05f tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.090718 Oct 20 09:08:34.111965 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 538/1074] 10.222.0.36 () {58 vars in 1149 bytes} [Thu Oct 20 09:08:34 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 693 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:08:34.126161 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2d2c7fae-2b8b-4c87-bb6e-50ca2cd77f10 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afb8c60d0>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:34.215789 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2d2c7fae-2b8b-4c87-bb6e-50ca2cd77f10 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.093104 Oct 20 09:08:34.226787 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 537/1075] 10.222.0.36 () {58 vars in 1134 bytes} [Thu Oct 20 09:08:34 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:34.237268 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bf94e584-d929-4eda-9603-527cbd28c112 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afb91ddc0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:34.338986 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bf94e584-d929-4eda-9603-527cbd28c112 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.110412 Oct 20 09:08:34.339589 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 539/1076] 10.222.0.36 () {58 vars in 1167 bytes} [Thu Oct 20 09:08:34 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:08:34.354167 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-34441f5f-c2e5-4c06-9f3a-d582ede9dade tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afb81de50>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:34.439100 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-34441f5f-c2e5-4c06-9f3a-d582ede9dade tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.088635 Oct 20 09:08:34.439762 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 538/1077] 10.222.0.36 () {58 vars in 1170 bytes} [Thu Oct 20 09:08:34 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 693 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:08:34.453701 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-411f8dfe-3442-45ba-b9e5-3e9cbbb94972 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afbab34c0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:34.539887 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-411f8dfe-3442-45ba-b9e5-3e9cbbb94972 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.089897 Oct 20 09:08:34.540486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 540/1078] 10.222.0.36 () {58 vars in 1155 bytes} [Thu Oct 20 09:08:34 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:08:34.551126 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:34.552194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=65,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:08:34.556040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c425ff09-0038-4b80-88c7-9bec38e543c2 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afb9e48b0>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:34.649920 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c425ff09-0038-4b80-88c7-9bec38e543c2 tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.099519 Oct 20 09:08:34.650500 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 539/1079] 10.222.0.36 () {58 vars in 1188 bytes} [Thu Oct 20 09:08:34 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:08:34.664445 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-acce525a-c42e-4347-99e2-f1378282512c tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] Calling method '.version_select at 0x7f5afbac75e0>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:08:34.759213 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-acce525a-c42e-4347-99e2-f1378282512c tempest-VolumesTestJSON-816004724 tempest-VolumesTestJSON-816004724-project] 10.222.0.36 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.099026 Oct 20 09:08:34.759792 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 541/1080] 10.222.0.36 () {58 vars in 1155 bytes} [Thu Oct 20 09:08:34 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:09:03.665067 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1465749814", "imageRef": "01db162c-52be-4229-b8fd-232aa961844d", "flavorRef": "452"}} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:09:03.715760 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.network.neutron [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] validate_networks() for None {{(pid=119068) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:09:04.000475 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Flavor limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:09:04.000895 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Image limits 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:09:04.001435 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Flavor pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:09:04.001828 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Image pref 0:0:0 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:09:04.002234 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.virt.hardware [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119068) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:09:04.012021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:04.012497 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:04.012925 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:04.013640 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:04.014083 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:04.014514 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:04.030490 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Getting quotas for project 0ce71f46acb24cd899e0d08974c825ab. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:09:04.034790 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.quota [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Getting quotas for user 62af90b699ca42a4aa0ac8f0edfdfd31 and project 0ce71f46acb24cd899e0d08974c825ab. Resources: {'cores', 'instances', 'ram'} {{(pid=119068) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:09:04.047646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Going to run 1 instances... {{(pid=119068) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:09:04.050013 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='01db162c-52be-4229-b8fd-232aa961844d',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119068) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:09:04.091699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR oslo.messaging._drivers.impl_rabbit [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [8e02c951-e228-47bd-b5ba-cdc75fa4bdcf] AMQP server on 10.222.0.36:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 20 09:09:05.099972 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO oslo.messaging._drivers.impl_rabbit [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [8e02c951-e228-47bd-b5ba-cdc75fa4bdcf] Reconnected to AMQP server on 10.222.0.36:5672 via [amqp] client with port 34294. Oct 20 09:09:05.104079 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-67cc122a-2ec6-438c-a799-053e0bfc4aaf tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.444216 Oct 20 09:09:05.104734 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 540/1081] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:09:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 1446 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:09:05.116486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-88577db3-30bc-46ba-96ae-f3dd658548fd tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:05.117212 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-88577db3-30bc-46ba-96ae-f3dd658548fd tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:05.174058 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-88577db3-30bc-46ba-96ae-f3dd658548fd tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:05.180920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-88577db3-30bc-46ba-96ae-f3dd658548fd tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1215 microversion: 2.1 time: 0.069076 Oct 20 09:09:05.181491 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 542/1082] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:05 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1215 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:06.204898 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3692508b-b20d-4d0a-a85d-832488edf3b7 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:06.205512 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3692508b-b20d-4d0a-a85d-832488edf3b7 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:06.211974 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3692508b-b20d-4d0a-a85d-832488edf3b7 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:06.212400 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3692508b-b20d-4d0a-a85d-832488edf3b7 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:06.212834 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3692508b-b20d-4d0a-a85d-832488edf3b7 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:06.213733 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3692508b-b20d-4d0a-a85d-832488edf3b7 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:06.214359 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3692508b-b20d-4d0a-a85d-832488edf3b7 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:06.215227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3692508b-b20d-4d0a-a85d-832488edf3b7 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:06.276779 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3692508b-b20d-4d0a-a85d-832488edf3b7 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:06.284872 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3692508b-b20d-4d0a-a85d-832488edf3b7 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1267 microversion: 2.1 time: 0.084522 Oct 20 09:09:06.285434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 541/1083] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:06 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1267 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:07.308783 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3fb5a04c-9b06-4588-ac99-c4d43041fab3 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:07.309393 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3fb5a04c-9b06-4588-ac99-c4d43041fab3 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:07.316515 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3fb5a04c-9b06-4588-ac99-c4d43041fab3 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:07.316988 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3fb5a04c-9b06-4588-ac99-c4d43041fab3 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:07.317419 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3fb5a04c-9b06-4588-ac99-c4d43041fab3 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:07.318105 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3fb5a04c-9b06-4588-ac99-c4d43041fab3 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:07.318581 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3fb5a04c-9b06-4588-ac99-c4d43041fab3 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:07.319002 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3fb5a04c-9b06-4588-ac99-c4d43041fab3 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:07.383874 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3fb5a04c-9b06-4588-ac99-c4d43041fab3 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:07.392657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3fb5a04c-9b06-4588-ac99-c4d43041fab3 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1273 microversion: 2.1 time: 0.087738 Oct 20 09:09:07.404948 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 543/1084] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:07 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1273 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:08.417007 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-2802bdf6-242a-4729-a223-442d7f756b79 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:08.417629 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-2802bdf6-242a-4729-a223-442d7f756b79 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:08.425037 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2802bdf6-242a-4729-a223-442d7f756b79 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:08.425524 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2802bdf6-242a-4729-a223-442d7f756b79 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:08.425965 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2802bdf6-242a-4729-a223-442d7f756b79 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:08.426697 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2802bdf6-242a-4729-a223-442d7f756b79 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:08.427153 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2802bdf6-242a-4729-a223-442d7f756b79 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:08.427581 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-2802bdf6-242a-4729-a223-442d7f756b79 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:08.558768 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-2802bdf6-242a-4729-a223-442d7f756b79 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:08.567210 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-2802bdf6-242a-4729-a223-442d7f756b79 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1315 microversion: 2.1 time: 0.154076 Oct 20 09:09:08.568065 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 542/1085] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:08 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:09.591292 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5851d448-e86e-45ae-b886-124d2dd961d5 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:09.591981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5851d448-e86e-45ae-b886-124d2dd961d5 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:09.599122 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5851d448-e86e-45ae-b886-124d2dd961d5 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:09.599755 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5851d448-e86e-45ae-b886-124d2dd961d5 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:09.600240 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5851d448-e86e-45ae-b886-124d2dd961d5 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:09.601122 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5851d448-e86e-45ae-b886-124d2dd961d5 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:09.601580 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5851d448-e86e-45ae-b886-124d2dd961d5 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:09.602136 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5851d448-e86e-45ae-b886-124d2dd961d5 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:09.731471 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5851d448-e86e-45ae-b886-124d2dd961d5 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:09.739857 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5851d448-e86e-45ae-b886-124d2dd961d5 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1315 microversion: 2.1 time: 0.152650 Oct 20 09:09:09.740418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 544/1086] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:09 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:10.634883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-864d6285-c004-4c24-8bcc-e85176831b20 req-d6535896-0c00-4d95-bc2c-043cb1ad1547 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d3d3c445-b406-4bc4-873d-abe3c68c25bf", "tag": "9019ed23-6243-4781-867c-a05083082422"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:09:10.645471 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-864d6285-c004-4c24-8bcc-e85176831b20 req-d6535896-0c00-4d95-bc2c-043cb1ad1547 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:10.645998 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-864d6285-c004-4c24-8bcc-e85176831b20 req-d6535896-0c00-4d95-bc2c-043cb1ad1547 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:10.646378 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-864d6285-c004-4c24-8bcc-e85176831b20 req-d6535896-0c00-4d95-bc2c-043cb1ad1547 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:10.658245 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-864d6285-c004-4c24-8bcc-e85176831b20 req-d6535896-0c00-4d95-bc2c-043cb1ad1547 service nova] Creating event network-changed:9019ed23-6243-4781-867c-a05083082422 for instance d3d3c445-b406-4bc4-873d-abe3c68c25bf on hv-cinder-maste Oct 20 09:09:10.663204 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-864d6285-c004-4c24-8bcc-e85176831b20 req-d6535896-0c00-4d95-bc2c-043cb1ad1547 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032083 Oct 20 09:09:10.663783 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 543/1087] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:09:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:09:10.763398 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-efcede78-ed10-418f-b934-893046ac982c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:10.764670 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-efcede78-ed10-418f-b934-893046ac982c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:10.771192 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-efcede78-ed10-418f-b934-893046ac982c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:10.771677 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-efcede78-ed10-418f-b934-893046ac982c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:10.773324 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-efcede78-ed10-418f-b934-893046ac982c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:10.773324 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-efcede78-ed10-418f-b934-893046ac982c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:10.773324 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-efcede78-ed10-418f-b934-893046ac982c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:10.773648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-efcede78-ed10-418f-b934-893046ac982c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:10.906368 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-efcede78-ed10-418f-b934-893046ac982c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:10.914031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-efcede78-ed10-418f-b934-893046ac982c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1315 microversion: 2.1 time: 0.154600 Oct 20 09:09:10.914681 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 545/1088] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:10 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:11.939424 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-22e15263-a36e-42e6-b618-35b5196f1117 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:11.940108 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-22e15263-a36e-42e6-b618-35b5196f1117 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:11.946978 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22e15263-a36e-42e6-b618-35b5196f1117 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:11.948767 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22e15263-a36e-42e6-b618-35b5196f1117 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:11.949212 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22e15263-a36e-42e6-b618-35b5196f1117 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:11.949899 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22e15263-a36e-42e6-b618-35b5196f1117 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:11.950347 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22e15263-a36e-42e6-b618-35b5196f1117 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:11.950799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-22e15263-a36e-42e6-b618-35b5196f1117 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:12.123617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-22e15263-a36e-42e6-b618-35b5196f1117 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:12.131283 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-22e15263-a36e-42e6-b618-35b5196f1117 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1315 microversion: 2.1 time: 0.196562 Oct 20 09:09:12.131852 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 544/1089] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:11 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1315 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:13.157390 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-60c4d811-773e-4065-a9a2-f3be5f43ad3c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:13.157992 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-60c4d811-773e-4065-a9a2-f3be5f43ad3c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:13.164564 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-60c4d811-773e-4065-a9a2-f3be5f43ad3c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:13.165042 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-60c4d811-773e-4065-a9a2-f3be5f43ad3c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:13.165473 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-60c4d811-773e-4065-a9a2-f3be5f43ad3c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:13.166163 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-60c4d811-773e-4065-a9a2-f3be5f43ad3c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:13.166643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-60c4d811-773e-4065-a9a2-f3be5f43ad3c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:13.167063 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-60c4d811-773e-4065-a9a2-f3be5f43ad3c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:13.298097 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-60c4d811-773e-4065-a9a2-f3be5f43ad3c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:13.305672 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-60c4d811-773e-4065-a9a2-f3be5f43ad3c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1315 microversion: 2.1 time: 0.152197 Oct 20 09:09:13.306237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 546/1090] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:13 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1315 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:14.329460 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-996c578e-63bd-469a-b860-59d9a31b74be tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:14.330053 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-996c578e-63bd-469a-b860-59d9a31b74be tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:14.332577 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:09:14.333431 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=6 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:09:14.338892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-996c578e-63bd-469a-b860-59d9a31b74be tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:14.339373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-996c578e-63bd-469a-b860-59d9a31b74be tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:14.340097 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-996c578e-63bd-469a-b860-59d9a31b74be tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:14.340778 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-996c578e-63bd-469a-b860-59d9a31b74be tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:14.341222 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-996c578e-63bd-469a-b860-59d9a31b74be tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:14.341671 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-996c578e-63bd-469a-b860-59d9a31b74be tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:14.471576 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-996c578e-63bd-469a-b860-59d9a31b74be tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:14.479329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-996c578e-63bd-469a-b860-59d9a31b74be tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1315 microversion: 2.1 time: 0.154357 Oct 20 09:09:14.479897 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 545/1091] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:14 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1315 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:14.677324 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1179999262", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1088348766", "security_groups": [{"name": "tempest-secgroup-smoke-1673307534"}], "block_device_mapping_v2": [{"uuid": "50b2f954-d418-4702-b710-a0ad217c8b57", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "d2d4ae81-03f2-49c0-8ca3-a270479acf30"}]}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:09:14.789506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.network.neutron [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] validate_networks() for [('d2d4ae81-03f2-49c0-8ca3-a270479acf30', None, None, None, None, None)] {{(pid=119067) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 20 09:09:15.146045 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Flavor limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 20 09:09:15.146407 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Image limits 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 20 09:09:15.146903 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Flavor pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 20 09:09:15.147308 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Image pref 0:0:0 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 20 09:09:15.147711 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.virt.hardware [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119067) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 20 09:09:15.156357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:15.156830 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:15.157259 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:15.157975 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:15.158452 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:15.158917 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:15.175032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Getting quotas for project 98827dbef155480b9f26792b70014b4b. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 20 09:09:15.178853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.quota [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Getting quotas for user 65ca2eae6db24bb094afaacdfb5c1980 and project 98827dbef155480b9f26792b70014b4b. Resources: {'cores', 'instances', 'ram'} {{(pid=119067) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 20 09:09:15.237270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Going to run 1 instances... {{(pid=119067) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 20 09:09:15.239395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='50b2f954-d418-4702-b710-a0ad217c8b57',volume_size=None,volume_type=None)] {{(pid=119067) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 20 09:09:15.369457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5ef17dc9-6de1-47bd-8511-df172ada6031 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.695958 Oct 20 09:09:15.370327 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 547/1092] 10.222.0.36 () {60 vars in 1147 bytes} [Thu Oct 20 09:09:14 2022] POST /compute/v2.1/servers => generated 406 bytes in 697 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 20 09:09:15.381748 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c84ba2c7-273a-43c3-a929-2dce09618421 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:15.382821 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c84ba2c7-273a-43c3-a929-2dce09618421 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:15.436018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c84ba2c7-273a-43c3-a929-2dce09618421 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:15.444219 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c84ba2c7-273a-43c3-a929-2dce09618421 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1095 microversion: 2.1 time: 0.066291 Oct 20 09:09:15.444791 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 546/1093] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:15 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1095 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:09:15.501731 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-d30db6d8-9247-416c-a6ba-f2cf2d20eaf4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:15.502395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-d30db6d8-9247-416c-a6ba-f2cf2d20eaf4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:15.508548 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d30db6d8-9247-416c-a6ba-f2cf2d20eaf4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:15.509034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d30db6d8-9247-416c-a6ba-f2cf2d20eaf4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:15.509470 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d30db6d8-9247-416c-a6ba-f2cf2d20eaf4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:15.510162 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d30db6d8-9247-416c-a6ba-f2cf2d20eaf4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:15.510647 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d30db6d8-9247-416c-a6ba-f2cf2d20eaf4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:15.511070 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-d30db6d8-9247-416c-a6ba-f2cf2d20eaf4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:15.646227 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-d30db6d8-9247-416c-a6ba-f2cf2d20eaf4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:15.654517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-d30db6d8-9247-416c-a6ba-f2cf2d20eaf4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1315 microversion: 2.1 time: 0.157146 Oct 20 09:09:15.655039 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 548/1094] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:15 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1315 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:16.246002 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-6dc5ae38-6344-4385-849e-be742192b545 req-b8145573-bee4-46ea-886b-dcca9a9304c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d3d3c445-b406-4bc4-873d-abe3c68c25bf", "name": "network-vif-plugged", "status": "completed", "tag": "9019ed23-6243-4781-867c-a05083082422"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:09:16.256781 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6dc5ae38-6344-4385-849e-be742192b545 req-b8145573-bee4-46ea-886b-dcca9a9304c1 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:16.257266 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6dc5ae38-6344-4385-849e-be742192b545 req-b8145573-bee4-46ea-886b-dcca9a9304c1 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:16.257691 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-6dc5ae38-6344-4385-849e-be742192b545 req-b8145573-bee4-46ea-886b-dcca9a9304c1 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:16.268467 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-6dc5ae38-6344-4385-849e-be742192b545 req-b8145573-bee4-46ea-886b-dcca9a9304c1 service nova] Creating event network-vif-plugged:9019ed23-6243-4781-867c-a05083082422 for instance d3d3c445-b406-4bc4-873d-abe3c68c25bf on hv-cinder-maste Oct 20 09:09:16.273414 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-6dc5ae38-6344-4385-849e-be742192b545 req-b8145573-bee4-46ea-886b-dcca9a9304c1 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031119 Oct 20 09:09:16.274029 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 547/1095] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:09:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:09:16.466268 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f25112df-aec1-4a42-a031-2dff8679fd4d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:16.466904 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f25112df-aec1-4a42-a031-2dff8679fd4d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:16.473939 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f25112df-aec1-4a42-a031-2dff8679fd4d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:16.474412 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f25112df-aec1-4a42-a031-2dff8679fd4d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:16.474869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f25112df-aec1-4a42-a031-2dff8679fd4d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:16.475543 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f25112df-aec1-4a42-a031-2dff8679fd4d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:16.475991 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f25112df-aec1-4a42-a031-2dff8679fd4d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:16.476408 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f25112df-aec1-4a42-a031-2dff8679fd4d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:16.536669 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f25112df-aec1-4a42-a031-2dff8679fd4d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:16.545354 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f25112df-aec1-4a42-a031-2dff8679fd4d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1193 microversion: 2.1 time: 0.083094 Oct 20 09:09:16.546131 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 549/1096] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:16 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1193 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:16.678822 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-6420f00a-8207-4ff6-be11-0f1a458ad64c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:16.679529 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-6420f00a-8207-4ff6-be11-0f1a458ad64c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:16.687051 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6420f00a-8207-4ff6-be11-0f1a458ad64c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:16.687810 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6420f00a-8207-4ff6-be11-0f1a458ad64c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:16.688251 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6420f00a-8207-4ff6-be11-0f1a458ad64c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:16.688965 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6420f00a-8207-4ff6-be11-0f1a458ad64c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:16.689579 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6420f00a-8207-4ff6-be11-0f1a458ad64c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:16.690052 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-6420f00a-8207-4ff6-be11-0f1a458ad64c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:16.950967 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-6420f00a-8207-4ff6-be11-0f1a458ad64c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:16.958499 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-6420f00a-8207-4ff6-be11-0f1a458ad64c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1315 microversion: 2.1 time: 0.284744 Oct 20 09:09:16.959079 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 548/1097] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:16 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1315 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:17.568531 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2e00ca75-5b0a-427d-98d5-05bb4dcded74 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:17.569121 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2e00ca75-5b0a-427d-98d5-05bb4dcded74 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:17.575775 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e00ca75-5b0a-427d-98d5-05bb4dcded74 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:17.576505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e00ca75-5b0a-427d-98d5-05bb4dcded74 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:17.577588 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e00ca75-5b0a-427d-98d5-05bb4dcded74 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:17.578287 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e00ca75-5b0a-427d-98d5-05bb4dcded74 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:17.578828 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e00ca75-5b0a-427d-98d5-05bb4dcded74 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:17.579249 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2e00ca75-5b0a-427d-98d5-05bb4dcded74 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:17.640856 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2e00ca75-5b0a-427d-98d5-05bb4dcded74 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:17.648634 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2e00ca75-5b0a-427d-98d5-05bb4dcded74 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1211 microversion: 2.1 time: 0.083996 Oct 20 09:09:17.649292 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 550/1098] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:17 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1211 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:17.981209 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-58f5b7a5-d5d2-4965-a609-e387fd9bbf39 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:17.981822 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-58f5b7a5-d5d2-4965-a609-e387fd9bbf39 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:17.989910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58f5b7a5-d5d2-4965-a609-e387fd9bbf39 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:17.990856 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58f5b7a5-d5d2-4965-a609-e387fd9bbf39 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:17.991299 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58f5b7a5-d5d2-4965-a609-e387fd9bbf39 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:17.992001 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58f5b7a5-d5d2-4965-a609-e387fd9bbf39 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:17.992647 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58f5b7a5-d5d2-4965-a609-e387fd9bbf39 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:17.993075 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-58f5b7a5-d5d2-4965-a609-e387fd9bbf39 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:18.229902 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-58f5b7a5-d5d2-4965-a609-e387fd9bbf39 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:18.238219 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-58f5b7a5-d5d2-4965-a609-e387fd9bbf39 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1315 microversion: 2.1 time: 0.260899 Oct 20 09:09:18.238929 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 549/1099] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:17 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1315 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:18.629324 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-b58ffd2f-260b-4c39-b713-c194ca9be2fa req-540b8220-eead-41c8-9b70-d3a61e4eb2f2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "842be7a4-f260-42e7-b34e-5f4369630d36", "tag": "9d2af910-87ab-4f40-ac9b-a6a140aaca39"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:09:18.659013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-b58ffd2f-260b-4c39-b713-c194ca9be2fa req-540b8220-eead-41c8-9b70-d3a61e4eb2f2 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:18.659495 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-b58ffd2f-260b-4c39-b713-c194ca9be2fa req-540b8220-eead-41c8-9b70-d3a61e4eb2f2 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:18.659930 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-b58ffd2f-260b-4c39-b713-c194ca9be2fa req-540b8220-eead-41c8-9b70-d3a61e4eb2f2 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:18.674840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-86591c67-86a5-40af-a784-674d1a249065 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:18.675883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-86591c67-86a5-40af-a784-674d1a249065 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:18.678405 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-b58ffd2f-260b-4c39-b713-c194ca9be2fa req-540b8220-eead-41c8-9b70-d3a61e4eb2f2 service nova] Creating event network-changed:9d2af910-87ab-4f40-ac9b-a6a140aaca39 for instance 842be7a4-f260-42e7-b34e-5f4369630d36 on hv-cinder-maste Oct 20 09:09:18.684559 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86591c67-86a5-40af-a784-674d1a249065 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:18.685078 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86591c67-86a5-40af-a784-674d1a249065 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:18.685459 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86591c67-86a5-40af-a784-674d1a249065 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:18.686203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86591c67-86a5-40af-a784-674d1a249065 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:18.686702 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86591c67-86a5-40af-a784-674d1a249065 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:18.687102 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-86591c67-86a5-40af-a784-674d1a249065 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:18.692213 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-b58ffd2f-260b-4c39-b713-c194ca9be2fa req-540b8220-eead-41c8-9b70-d3a61e4eb2f2 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066481 Oct 20 09:09:18.693029 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 551/1100] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:09:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:09:18.858180 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-86591c67-86a5-40af-a784-674d1a249065 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:18.866098 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-86591c67-86a5-40af-a784-674d1a249065 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.198188 Oct 20 09:09:18.866697 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 550/1101] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:18 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:19.261719 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-1079113d-f4dc-4d70-a330-f57baa8fc726 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:19.262330 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-1079113d-f4dc-4d70-a330-f57baa8fc726 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:19.268406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1079113d-f4dc-4d70-a330-f57baa8fc726 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:19.268883 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1079113d-f4dc-4d70-a330-f57baa8fc726 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:19.269323 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1079113d-f4dc-4d70-a330-f57baa8fc726 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:19.270008 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1079113d-f4dc-4d70-a330-f57baa8fc726 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:19.270457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1079113d-f4dc-4d70-a330-f57baa8fc726 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:19.270903 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-1079113d-f4dc-4d70-a330-f57baa8fc726 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:19.402608 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-1079113d-f4dc-4d70-a330-f57baa8fc726 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:19.410777 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-1079113d-f4dc-4d70-a330-f57baa8fc726 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1465 microversion: 2.1 time: 0.153337 Oct 20 09:09:19.411341 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 552/1102] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:19 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1465 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:19.457598 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-497f0e0a-f575-49b1-92ee-34e4b95f409a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:19.458091 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-497f0e0a-f575-49b1-92ee-34e4b95f409a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:19.464373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-497f0e0a-f575-49b1-92ee-34e4b95f409a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:19.465106 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-497f0e0a-f575-49b1-92ee-34e4b95f409a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:19.465538 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-497f0e0a-f575-49b1-92ee-34e4b95f409a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:19.466216 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-497f0e0a-f575-49b1-92ee-34e4b95f409a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:19.466703 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-497f0e0a-f575-49b1-92ee-34e4b95f409a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:19.467126 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-497f0e0a-f575-49b1-92ee-34e4b95f409a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:19.494464 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-497f0e0a-f575-49b1-92ee-34e4b95f409a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:09:19.571360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-497f0e0a-f575-49b1-92ee-34e4b95f409a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "DELETE /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 204 len: 0 microversion: 2.1 time: 0.117186 Oct 20 09:09:19.572383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 551/1103] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:09:19 2022] DELETE /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:09:19.581077 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-081d1515-1713-40a6-a8a0-f1048e515d71 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:19.581719 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-081d1515-1713-40a6-a8a0-f1048e515d71 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:19.587624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-081d1515-1713-40a6-a8a0-f1048e515d71 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:19.588107 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-081d1515-1713-40a6-a8a0-f1048e515d71 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:19.588538 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-081d1515-1713-40a6-a8a0-f1048e515d71 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:19.589232 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-081d1515-1713-40a6-a8a0-f1048e515d71 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:19.589680 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-081d1515-1713-40a6-a8a0-f1048e515d71 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:19.590361 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-081d1515-1713-40a6-a8a0-f1048e515d71 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:19.719118 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-081d1515-1713-40a6-a8a0-f1048e515d71 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:19.727092 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-081d1515-1713-40a6-a8a0-f1048e515d71 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1471 microversion: 2.1 time: 0.150759 Oct 20 09:09:19.727837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 553/1104] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:19 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1471 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:19.890559 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5ee27bca-9cb3-4ea6-8bfd-81d94de68dea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:19.891282 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5ee27bca-9cb3-4ea6-8bfd-81d94de68dea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:19.897565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee27bca-9cb3-4ea6-8bfd-81d94de68dea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:19.898061 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee27bca-9cb3-4ea6-8bfd-81d94de68dea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:19.898639 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee27bca-9cb3-4ea6-8bfd-81d94de68dea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:19.899212 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee27bca-9cb3-4ea6-8bfd-81d94de68dea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:19.899673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee27bca-9cb3-4ea6-8bfd-81d94de68dea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:19.900130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee27bca-9cb3-4ea6-8bfd-81d94de68dea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:20.057961 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5ee27bca-9cb3-4ea6-8bfd-81d94de68dea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:20.066548 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5ee27bca-9cb3-4ea6-8bfd-81d94de68dea tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.180748 Oct 20 09:09:20.067203 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 552/1105] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:19 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:20.750889 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-3cab0690-fed6-45ef-a05e-a27ebf9ced40 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:20.751410 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-3cab0690-fed6-45ef-a05e-a27ebf9ced40 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:20.757754 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3cab0690-fed6-45ef-a05e-a27ebf9ced40 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:20.758236 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3cab0690-fed6-45ef-a05e-a27ebf9ced40 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:20.758741 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3cab0690-fed6-45ef-a05e-a27ebf9ced40 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:20.759428 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3cab0690-fed6-45ef-a05e-a27ebf9ced40 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:20.759882 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3cab0690-fed6-45ef-a05e-a27ebf9ced40 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:20.760307 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-3cab0690-fed6-45ef-a05e-a27ebf9ced40 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:20.888427 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-3cab0690-fed6-45ef-a05e-a27ebf9ced40 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:20.896680 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-3cab0690-fed6-45ef-a05e-a27ebf9ced40 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1471 microversion: 2.1 time: 0.149762 Oct 20 09:09:20.897246 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 554/1106] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:20 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1471 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:21.089630 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-06096a9f-229d-4708-b3d4-d5fc33aae419 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:21.090565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-06096a9f-229d-4708-b3d4-d5fc33aae419 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:21.101418 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06096a9f-229d-4708-b3d4-d5fc33aae419 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:21.101418 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06096a9f-229d-4708-b3d4-d5fc33aae419 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:21.101418 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06096a9f-229d-4708-b3d4-d5fc33aae419 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:21.101418 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06096a9f-229d-4708-b3d4-d5fc33aae419 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:21.101773 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06096a9f-229d-4708-b3d4-d5fc33aae419 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:21.101773 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-06096a9f-229d-4708-b3d4-d5fc33aae419 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:21.410663 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-06096a9f-229d-4708-b3d4-d5fc33aae419 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:21.418274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-06096a9f-229d-4708-b3d4-d5fc33aae419 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.332523 Oct 20 09:09:21.418873 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 553/1107] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:21 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:21.812698 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-dbf32338-75dd-4a44-80c9-f01d0b27d7e4 req-6844df1a-aa29-4945-b293-4c9468dc6bdb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d3d3c445-b406-4bc4-873d-abe3c68c25bf", "name": "network-vif-deleted", "tag": "9019ed23-6243-4781-867c-a05083082422"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:09:21.824221 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dbf32338-75dd-4a44-80c9-f01d0b27d7e4 req-6844df1a-aa29-4945-b293-4c9468dc6bdb service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:21.824931 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dbf32338-75dd-4a44-80c9-f01d0b27d7e4 req-6844df1a-aa29-4945-b293-4c9468dc6bdb service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:21.825357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-dbf32338-75dd-4a44-80c9-f01d0b27d7e4 req-6844df1a-aa29-4945-b293-4c9468dc6bdb service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:21.837140 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-dbf32338-75dd-4a44-80c9-f01d0b27d7e4 req-6844df1a-aa29-4945-b293-4c9468dc6bdb service nova] Creating event network-vif-deleted:9019ed23-6243-4781-867c-a05083082422 for instance d3d3c445-b406-4bc4-873d-abe3c68c25bf on hv-cinder-maste Oct 20 09:09:21.842818 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-dbf32338-75dd-4a44-80c9-f01d0b27d7e4 req-6844df1a-aa29-4945-b293-4c9468dc6bdb service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033134 Oct 20 09:09:21.843426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 555/1108] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:09:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:09:21.921809 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3d819d69-1367-46bd-8f12-2d7e254b01cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:21.922418 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3d819d69-1367-46bd-8f12-2d7e254b01cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:21.928362 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3d819d69-1367-46bd-8f12-2d7e254b01cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:21.928846 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3d819d69-1367-46bd-8f12-2d7e254b01cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:21.929285 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3d819d69-1367-46bd-8f12-2d7e254b01cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:21.929971 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3d819d69-1367-46bd-8f12-2d7e254b01cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:21.930427 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3d819d69-1367-46bd-8f12-2d7e254b01cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:21.930927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3d819d69-1367-46bd-8f12-2d7e254b01cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:21.995046 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3d819d69-1367-46bd-8f12-2d7e254b01cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:22.002704 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3d819d69-1367-46bd-8f12-2d7e254b01cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1429 microversion: 2.1 time: 0.085303 Oct 20 09:09:22.003274 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 554/1109] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:21 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1429 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:22.442947 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b49e822b-1316-4714-8949-3a2a0a833ef3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:22.443657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b49e822b-1316-4714-8949-3a2a0a833ef3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:22.449709 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b49e822b-1316-4714-8949-3a2a0a833ef3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:22.450182 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b49e822b-1316-4714-8949-3a2a0a833ef3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:22.450640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b49e822b-1316-4714-8949-3a2a0a833ef3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:22.451315 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b49e822b-1316-4714-8949-3a2a0a833ef3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:22.452310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b49e822b-1316-4714-8949-3a2a0a833ef3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:22.452727 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b49e822b-1316-4714-8949-3a2a0a833ef3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:22.584986 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b49e822b-1316-4714-8949-3a2a0a833ef3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:22.592941 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b49e822b-1316-4714-8949-3a2a0a833ef3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.154136 Oct 20 09:09:22.593503 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 556/1110] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:22 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:09:23.025790 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-da61205b-f889-4aa4-8e49-3628eb145a10 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:23.026406 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-da61205b-f889-4aa4-8e49-3628eb145a10 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:23.032804 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da61205b-f889-4aa4-8e49-3628eb145a10 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:23.033318 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da61205b-f889-4aa4-8e49-3628eb145a10 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:23.033752 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da61205b-f889-4aa4-8e49-3628eb145a10 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:23.034434 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da61205b-f889-4aa4-8e49-3628eb145a10 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:23.034913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da61205b-f889-4aa4-8e49-3628eb145a10 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:23.035335 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-da61205b-f889-4aa4-8e49-3628eb145a10 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:23.095408 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-da61205b-f889-4aa4-8e49-3628eb145a10 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:23.102940 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-da61205b-f889-4aa4-8e49-3628eb145a10 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1429 microversion: 2.1 time: 0.081006 Oct 20 09:09:23.103505 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 555/1111] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:23 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1429 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:23.615054 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ba37e294-bd87-4ddf-a4a6-1823713e6208 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:23.615655 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ba37e294-bd87-4ddf-a4a6-1823713e6208 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:23.621987 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ba37e294-bd87-4ddf-a4a6-1823713e6208 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:23.622464 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ba37e294-bd87-4ddf-a4a6-1823713e6208 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:23.622924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ba37e294-bd87-4ddf-a4a6-1823713e6208 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:23.623667 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ba37e294-bd87-4ddf-a4a6-1823713e6208 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:23.624109 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ba37e294-bd87-4ddf-a4a6-1823713e6208 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:23.624526 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ba37e294-bd87-4ddf-a4a6-1823713e6208 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:23.750445 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ba37e294-bd87-4ddf-a4a6-1823713e6208 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:23.758952 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ba37e294-bd87-4ddf-a4a6-1823713e6208 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.147793 Oct 20 09:09:23.759653 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 557/1112] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:23 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:24.127166 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-839aa4b7-a416-449f-9ebb-9ccd9de6ebc0 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:24.127799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-839aa4b7-a416-449f-9ebb-9ccd9de6ebc0 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:24.134138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-839aa4b7-a416-449f-9ebb-9ccd9de6ebc0 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:24.134596 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-839aa4b7-a416-449f-9ebb-9ccd9de6ebc0 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:24.135034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-839aa4b7-a416-449f-9ebb-9ccd9de6ebc0 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:24.135714 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-839aa4b7-a416-449f-9ebb-9ccd9de6ebc0 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:24.136165 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-839aa4b7-a416-449f-9ebb-9ccd9de6ebc0 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:24.136595 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-839aa4b7-a416-449f-9ebb-9ccd9de6ebc0 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:24.197219 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-839aa4b7-a416-449f-9ebb-9ccd9de6ebc0 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:24.205506 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-839aa4b7-a416-449f-9ebb-9ccd9de6ebc0 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1300 microversion: 2.1 time: 0.082757 Oct 20 09:09:24.206064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 556/1113] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:24 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1300 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:24.784677 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-5b685084-aeb7-40a4-8b27-bb6e4926feb8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:24.785203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-5b685084-aeb7-40a4-8b27-bb6e4926feb8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:24.791456 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b685084-aeb7-40a4-8b27-bb6e4926feb8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:24.791945 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b685084-aeb7-40a4-8b27-bb6e4926feb8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:24.792374 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b685084-aeb7-40a4-8b27-bb6e4926feb8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:24.793057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b685084-aeb7-40a4-8b27-bb6e4926feb8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:24.793499 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b685084-aeb7-40a4-8b27-bb6e4926feb8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:24.793918 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-5b685084-aeb7-40a4-8b27-bb6e4926feb8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:24.925488 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-5b685084-aeb7-40a4-8b27-bb6e4926feb8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:24.933196 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-5b685084-aeb7-40a4-8b27-bb6e4926feb8 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.153657 Oct 20 09:09:24.933768 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 558/1114] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:24 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:25.228715 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:25.229324 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:25.235841 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:25.236344 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:25.236764 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:25.237453 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:25.237906 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:25.238334 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:25.270570 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:25.271041 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:25.271474 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:25.271913 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lazy-loading 'fault' on Instance uuid d3d3c445-b406-4bc4-873d-abe3c68c25bf {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:09:25.274678 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:09:25.312692 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:25.320229 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-7dc84b3f-7015-42b0-8e33-eb0d1ac5eb7e tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1305 microversion: 2.1 time: 0.095376 Oct 20 09:09:25.320812 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 557/1115] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:25 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1305 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:25.956250 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4aaaa4bc-cbc3-4ba8-a74e-0f88a189cf8a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:25.956851 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4aaaa4bc-cbc3-4ba8-a74e-0f88a189cf8a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:25.958830 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:09:25.959526 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell0:SELECT=8 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:09:25.965865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4aaaa4bc-cbc3-4ba8-a74e-0f88a189cf8a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:25.966335 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4aaaa4bc-cbc3-4ba8-a74e-0f88a189cf8a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:25.966797 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4aaaa4bc-cbc3-4ba8-a74e-0f88a189cf8a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:25.968111 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4aaaa4bc-cbc3-4ba8-a74e-0f88a189cf8a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:25.968553 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4aaaa4bc-cbc3-4ba8-a74e-0f88a189cf8a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:25.968966 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4aaaa4bc-cbc3-4ba8-a74e-0f88a189cf8a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:26.096426 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4aaaa4bc-cbc3-4ba8-a74e-0f88a189cf8a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:26.104022 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4aaaa4bc-cbc3-4ba8-a74e-0f88a189cf8a tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.152175 Oct 20 09:09:26.104560 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 559/1116] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:25 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:26.344581 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:26.345184 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:26.346918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell0:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:09:26.352600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:26.353099 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:26.353535 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:26.354212 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:26.354684 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:26.355103 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:26.386131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:26.386642 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:26.387074 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:26.387509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lazy-loading 'fault' on Instance uuid d3d3c445-b406-4bc4-873d-abe3c68c25bf {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:09:26.429960 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:26.437467 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-af0bc854-bbe1-4388-a5e2-eeae7fbdb39a tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1305 microversion: 2.1 time: 0.097536 Oct 20 09:09:26.438027 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 558/1117] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:26 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1305 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:27.126785 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-2ea614d8-5108-4187-bcfd-5f405972d0b4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:27.127391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-2ea614d8-5108-4187-bcfd-5f405972d0b4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:27.133467 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2ea614d8-5108-4187-bcfd-5f405972d0b4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:27.133942 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2ea614d8-5108-4187-bcfd-5f405972d0b4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:27.134376 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2ea614d8-5108-4187-bcfd-5f405972d0b4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:27.135081 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2ea614d8-5108-4187-bcfd-5f405972d0b4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:27.135528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2ea614d8-5108-4187-bcfd-5f405972d0b4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:27.135950 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-2ea614d8-5108-4187-bcfd-5f405972d0b4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:27.261670 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-2ea614d8-5108-4187-bcfd-5f405972d0b4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:27.269283 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-2ea614d8-5108-4187-bcfd-5f405972d0b4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.146474 Oct 20 09:09:27.269856 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 560/1118] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:27 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:27.460489 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:27.461095 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:27.467804 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:27.468286 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:27.468724 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:27.469410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:27.469858 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:27.470278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:27.501787 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:27.502293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:27.502765 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:27.503209 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lazy-loading 'fault' on Instance uuid d3d3c445-b406-4bc4-873d-abe3c68c25bf {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:09:27.541646 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:27.549754 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-403b01db-93de-4d57-b37f-a2e469b6a9cc tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1305 microversion: 2.1 time: 0.093176 Oct 20 09:09:27.550330 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 559/1119] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:27 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1305 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:28.292576 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a838fdaa-0b33-4b09-95fc-5fe53e526771 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:28.293190 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a838fdaa-0b33-4b09-95fc-5fe53e526771 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:28.300559 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a838fdaa-0b33-4b09-95fc-5fe53e526771 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:28.301096 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a838fdaa-0b33-4b09-95fc-5fe53e526771 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:28.301505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a838fdaa-0b33-4b09-95fc-5fe53e526771 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:28.302193 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a838fdaa-0b33-4b09-95fc-5fe53e526771 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:28.302704 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a838fdaa-0b33-4b09-95fc-5fe53e526771 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:28.303154 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a838fdaa-0b33-4b09-95fc-5fe53e526771 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:28.323391 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=100 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:09:28.439041 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a838fdaa-0b33-4b09-95fc-5fe53e526771 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:28.446754 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a838fdaa-0b33-4b09-95fc-5fe53e526771 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.158239 Oct 20 09:09:28.447324 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 561/1120] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:28 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:28.572886 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:28.573390 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:28.579705 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:28.580186 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:28.580615 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:28.581289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:28.581734 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:28.582153 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:28.612686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:28.613150 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:28.613579 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:28.614011 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lazy-loading 'fault' on Instance uuid d3d3c445-b406-4bc4-873d-abe3c68c25bf {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:09:28.652745 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:28.660667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-508b350c-4f91-49ac-ab97-dbc5687c90b1 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1305 microversion: 2.1 time: 0.091708 Oct 20 09:09:28.661224 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 560/1121] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:28 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1305 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:29.469405 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8f76634e-0252-4d70-a38e-7cce67338c43 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:29.469997 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8f76634e-0252-4d70-a38e-7cce67338c43 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:29.476375 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8f76634e-0252-4d70-a38e-7cce67338c43 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:29.476853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8f76634e-0252-4d70-a38e-7cce67338c43 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:29.477289 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8f76634e-0252-4d70-a38e-7cce67338c43 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:29.477966 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8f76634e-0252-4d70-a38e-7cce67338c43 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:29.478413 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8f76634e-0252-4d70-a38e-7cce67338c43 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:29.478869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8f76634e-0252-4d70-a38e-7cce67338c43 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:29.607269 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8f76634e-0252-4d70-a38e-7cce67338c43 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:29.615457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8f76634e-0252-4d70-a38e-7cce67338c43 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.149920 Oct 20 09:09:29.616028 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 562/1122] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:29 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:29.683560 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:29.684164 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:29.691238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:29.691718 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:29.692156 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:29.692833 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:29.693278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:29.693699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:29.723004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:29.723470 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:29.723899 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:29.724331 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lazy-loading 'fault' on Instance uuid d3d3c445-b406-4bc4-873d-abe3c68c25bf {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:09:29.764067 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:29.771572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-41f807e4-45cd-49d8-83f3-1a7713022fa4 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1305 microversion: 2.1 time: 0.091826 Oct 20 09:09:29.772121 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 561/1123] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:29 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1305 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:30.639603 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ea3dac59-1a95-4fa1-971f-1b918de1ee92 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:30.640901 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ea3dac59-1a95-4fa1-971f-1b918de1ee92 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:30.646952 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea3dac59-1a95-4fa1-971f-1b918de1ee92 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:30.647431 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea3dac59-1a95-4fa1-971f-1b918de1ee92 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:30.647868 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea3dac59-1a95-4fa1-971f-1b918de1ee92 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:30.648559 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea3dac59-1a95-4fa1-971f-1b918de1ee92 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:30.649031 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea3dac59-1a95-4fa1-971f-1b918de1ee92 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:30.649456 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea3dac59-1a95-4fa1-971f-1b918de1ee92 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:30.784546 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ea3dac59-1a95-4fa1-971f-1b918de1ee92 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:30.793472 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:30.794048 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ea3dac59-1a95-4fa1-971f-1b918de1ee92 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.159179 Oct 20 09:09:30.794847 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:30.794975 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 563/1124] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:30 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:30.802096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:30.802610 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:30.803309 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:30.804114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:30.804670 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:30.805370 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:30.838615 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:30.839222 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:30.839659 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:30.840363 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lazy-loading 'fault' on Instance uuid d3d3c445-b406-4bc4-873d-abe3c68c25bf {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:09:30.881217 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:30.888824 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-30d05beb-bdf2-4c5c-8a42-057c11f3d874 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1305 microversion: 2.1 time: 0.099189 Oct 20 09:09:30.889399 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 562/1125] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:30 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1305 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:09:31.817240 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f173237d-eba0-41f0-a2a6-4549e8119c38 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:31.817847 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f173237d-eba0-41f0-a2a6-4549e8119c38 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:31.824284 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f173237d-eba0-41f0-a2a6-4549e8119c38 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:31.824775 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f173237d-eba0-41f0-a2a6-4549e8119c38 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:31.825205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f173237d-eba0-41f0-a2a6-4549e8119c38 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:31.825947 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f173237d-eba0-41f0-a2a6-4549e8119c38 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:31.826410 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f173237d-eba0-41f0-a2a6-4549e8119c38 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:31.826841 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f173237d-eba0-41f0-a2a6-4549e8119c38 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:31.913194 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:31.913969 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:31.920493 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:31.920983 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:31.921425 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:31.922124 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:31.922606 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:31.923041 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:31.953862 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:31.954354 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:31.954821 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:31.955268 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lazy-loading 'fault' on Instance uuid d3d3c445-b406-4bc4-873d-abe3c68c25bf {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:09:31.957152 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f173237d-eba0-41f0-a2a6-4549e8119c38 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:31.965150 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f173237d-eba0-41f0-a2a6-4549e8119c38 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.151570 Oct 20 09:09:31.965921 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 564/1126] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:31 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:31.997260 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:32.004877 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-3588ccc6-0147-4136-9cb0-691da8ed3008 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1305 microversion: 2.1 time: 0.095606 Oct 20 09:09:32.005445 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 563/1127] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:31 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1305 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:32.988176 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-462ab1fe-a18d-4909-bf16-cffe0edb91d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:32.988712 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-462ab1fe-a18d-4909-bf16-cffe0edb91d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:32.995398 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-462ab1fe-a18d-4909-bf16-cffe0edb91d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:32.995874 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-462ab1fe-a18d-4909-bf16-cffe0edb91d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:32.996306 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-462ab1fe-a18d-4909-bf16-cffe0edb91d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:32.996987 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-462ab1fe-a18d-4909-bf16-cffe0edb91d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:32.997430 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-462ab1fe-a18d-4909-bf16-cffe0edb91d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:32.997846 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-462ab1fe-a18d-4909-bf16-cffe0edb91d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:33.028581 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:33.029329 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:33.035864 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:33.036232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:33.036676 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:33.037380 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:33.037864 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:33.038292 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:33.074994 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:33.076004 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:33.076402 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:33.076846 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lazy-loading 'fault' on Instance uuid d3d3c445-b406-4bc4-873d-abe3c68c25bf {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:09:33.118139 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:33.127750 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-5ee0165d-8952-4304-9ab0-7a25e96f6717 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1305 microversion: 2.1 time: 0.103025 Oct 20 09:09:33.128387 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 564/1128] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:33 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1305 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:09:33.134462 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-462ab1fe-a18d-4909-bf16-cffe0edb91d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:33.143946 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-462ab1fe-a18d-4909-bf16-cffe0edb91d2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.159770 Oct 20 09:09:33.144519 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 565/1129] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:32 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:34.151399 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:34.152012 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:34.154373 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:09:34.160361 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:34.160851 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:34.161293 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:34.161992 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:34.162452 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:34.162910 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:34.167266 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b9050ff9-fa30-463a-9455-d4553e04f923 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:34.167865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b9050ff9-fa30-463a-9455-d4553e04f923 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:34.174326 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b9050ff9-fa30-463a-9455-d4553e04f923 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:34.174847 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b9050ff9-fa30-463a-9455-d4553e04f923 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:34.175279 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b9050ff9-fa30-463a-9455-d4553e04f923 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:34.175965 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b9050ff9-fa30-463a-9455-d4553e04f923 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:34.176410 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b9050ff9-fa30-463a-9455-d4553e04f923 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:34.176831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b9050ff9-fa30-463a-9455-d4553e04f923 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:34.197175 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:34.198018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:34.198593 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:34.199054 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lazy-loading 'fault' on Instance uuid d3d3c445-b406-4bc4-873d-abe3c68c25bf {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:09:34.242885 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:34.250456 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c0257095-8c64-4841-9630-3560c9f3483c tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1305 microversion: 2.1 time: 0.103010 Oct 20 09:09:34.251059 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 565/1130] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:34 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1305 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:34.312297 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b9050ff9-fa30-463a-9455-d4553e04f923 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:34.322086 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b9050ff9-fa30-463a-9455-d4553e04f923 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.158675 Oct 20 09:09:34.322808 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 566/1131] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:34 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:09:35.273232 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:35.273836 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:35.280325 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:35.280806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:35.281238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:35.282417 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:35.282890 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:35.283314 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:35.313473 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:35.313942 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:35.314375 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:35.314839 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.objects.instance [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lazy-loading 'fault' on Instance uuid d3d3c445-b406-4bc4-873d-abe3c68c25bf {{(pid=119068) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:09:35.346440 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ea0eb174-93fe-409b-91bc-b4e50a5eef69 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:35.347070 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ea0eb174-93fe-409b-91bc-b4e50a5eef69 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:35.349381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=66,nova_api:INSERT=3 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:09:35.354638 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62af90b699ca42a4aa0ac8f0edfdfd31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0ce71f46acb24cd899e0d08974c825ab', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:35.354837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea0eb174-93fe-409b-91bc-b4e50a5eef69 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:35.355317 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea0eb174-93fe-409b-91bc-b4e50a5eef69 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:35.355746 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea0eb174-93fe-409b-91bc-b4e50a5eef69 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:35.356430 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea0eb174-93fe-409b-91bc-b4e50a5eef69 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:35.356872 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea0eb174-93fe-409b-91bc-b4e50a5eef69 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:35.357288 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ea0eb174-93fe-409b-91bc-b4e50a5eef69 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:35.365933 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-534ac486-be87-48eb-ad43-bbce28d05e99 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 200 len: 1305 microversion: 2.1 time: 0.093725 Oct 20 09:09:35.367140 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 566/1132] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:35 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 1305 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:35.492373 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ea0eb174-93fe-409b-91bc-b4e50a5eef69 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:35.500751 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ea0eb174-93fe-409b-91bc-b4e50a5eef69 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.158111 Oct 20 09:09:35.501455 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 567/1133] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:35 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:09:36.388127 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:36.388733 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] [instance: d3d3c445-b406-4bc4-873d-abe3c68c25bf] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:36.395307 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:36.395806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:36.396221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:36.396907 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:36.397360 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:36.397787 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:36.411016 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] HTTP exception thrown: Instance d3d3c445-b406-4bc4-873d-abe3c68c25bf could not be found. Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Request method failure captured: Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf HTTP/1.1 Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjURAuvaX6xFukhcNVUOyWr8MjX8F5fOoGeMbCmY8tSib9FVtdIgOpgBdxAncy2_PORUMQuhI1WrH3P9FcA9JCjSrySyrjRSjN2RE9bDlFpxhBpYoo0kbkRLXWWaJnv0Xp0QoVZiNi7UR-ubXmroDhJX3-3cMYCZKwcKA6Se3X_2yEuaA Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 0ce71f46acb24cd899e0d08974c825ab Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-VolumesNegativeTest-1146834603 Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:09:36.412635 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/0ce71f46acb24cd899e0d08974c825ab"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_0ce71f46acb24cd899e0d08974c825ab", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/0ce71f46acb24cd899e0d08974c825ab"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/0ce71f46acb24cd899e0d08974c825ab"}]}] Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-VolumesNegativeTest-1146834603 Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 0ce71f46acb24cd899e0d08974c825ab Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-VolumesNegativeTest-1146834603 Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-VolumesNegativeTest-1146834603-project Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 62af90b699ca42a4aa0ac8f0edfdfd31 Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-VolumesNegativeTest-1146834603-project Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance d3d3c445-b406-4bc4-873d-abe3c68c25bf could not be found. Oct 20 09:09:36.414356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': 'd3d3c445-b406-4bc4-873d-abe3c68c25bf'} Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d3d3c445-b406-4bc4-873d-abe3c68c25bf could not be found. Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:09:36.415569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d3d3c445-b406-4bc4-873d-abe3c68c25bf could not be found. Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d3d3c445-b406-4bc4-873d-abe3c68c25bf could not be found. Oct 20 09:09:36.417163 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:09:36.418617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] Returning 404 to user: Instance d3d3c445-b406-4bc4-873d-abe3c68c25bf could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:09:36.418617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-968fa725-fd76-494b-b2d5-8107c9dfe274 tempest-VolumesNegativeTest-1146834603 tempest-VolumesNegativeTest-1146834603-project] 10.222.0.36 "GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf" status: 404 len: 111 microversion: 2.1 time: 0.029764 Oct 20 09:09:36.418617 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 567/1134] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:36 2022] GET /compute/v2.1/servers/d3d3c445-b406-4bc4-873d-abe3c68c25bf => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:09:36.523244 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a41d6ed2-abb8-4ffb-a1b2-dc9dbc9c4bbc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:36.524348 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a41d6ed2-abb8-4ffb-a1b2-dc9dbc9c4bbc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:36.532079 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a41d6ed2-abb8-4ffb-a1b2-dc9dbc9c4bbc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:36.532555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a41d6ed2-abb8-4ffb-a1b2-dc9dbc9c4bbc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:36.533164 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a41d6ed2-abb8-4ffb-a1b2-dc9dbc9c4bbc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:36.533847 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a41d6ed2-abb8-4ffb-a1b2-dc9dbc9c4bbc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:36.534577 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a41d6ed2-abb8-4ffb-a1b2-dc9dbc9c4bbc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:36.534995 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a41d6ed2-abb8-4ffb-a1b2-dc9dbc9c4bbc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:36.677074 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a41d6ed2-abb8-4ffb-a1b2-dc9dbc9c4bbc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:36.685503 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a41d6ed2-abb8-4ffb-a1b2-dc9dbc9c4bbc tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.166020 Oct 20 09:09:36.686394 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 568/1135] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:36 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:37.712262 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b77ff18a-aab1-4812-b307-c36f809f92a2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:37.712878 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b77ff18a-aab1-4812-b307-c36f809f92a2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:37.719607 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b77ff18a-aab1-4812-b307-c36f809f92a2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:37.720092 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b77ff18a-aab1-4812-b307-c36f809f92a2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:37.720596 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b77ff18a-aab1-4812-b307-c36f809f92a2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:37.721242 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b77ff18a-aab1-4812-b307-c36f809f92a2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:37.721667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b77ff18a-aab1-4812-b307-c36f809f92a2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:37.722107 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b77ff18a-aab1-4812-b307-c36f809f92a2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:37.870208 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b77ff18a-aab1-4812-b307-c36f809f92a2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:37.878213 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b77ff18a-aab1-4812-b307-c36f809f92a2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.169751 Oct 20 09:09:37.878802 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 568/1136] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:37 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:38.746136 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-789911fb-34ab-4787-a659-95dc8ef0eac1 req-3a063af6-33aa-44ce-b82b-45dfb177370b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "842be7a4-f260-42e7-b34e-5f4369630d36", "name": "network-vif-plugged", "status": "completed", "tag": "9d2af910-87ab-4f40-ac9b-a6a140aaca39"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:09:38.757355 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-789911fb-34ab-4787-a659-95dc8ef0eac1 req-3a063af6-33aa-44ce-b82b-45dfb177370b service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:38.757840 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-789911fb-34ab-4787-a659-95dc8ef0eac1 req-3a063af6-33aa-44ce-b82b-45dfb177370b service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:38.759414 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-789911fb-34ab-4787-a659-95dc8ef0eac1 req-3a063af6-33aa-44ce-b82b-45dfb177370b service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:38.772916 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-789911fb-34ab-4787-a659-95dc8ef0eac1 req-3a063af6-33aa-44ce-b82b-45dfb177370b service nova] Creating event network-vif-plugged:9d2af910-87ab-4f40-ac9b-a6a140aaca39 for instance 842be7a4-f260-42e7-b34e-5f4369630d36 on hv-cinder-maste Oct 20 09:09:38.777908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-789911fb-34ab-4787-a659-95dc8ef0eac1 req-3a063af6-33aa-44ce-b82b-45dfb177370b service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035522 Oct 20 09:09:38.778720 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 569/1137] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:09:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:09:38.900559 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-de7504dc-c2b3-4672-8731-07e1e1a34342 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:38.901252 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-de7504dc-c2b3-4672-8731-07e1e1a34342 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:38.912410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de7504dc-c2b3-4672-8731-07e1e1a34342 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:38.912903 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de7504dc-c2b3-4672-8731-07e1e1a34342 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:38.913464 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de7504dc-c2b3-4672-8731-07e1e1a34342 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:38.914026 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de7504dc-c2b3-4672-8731-07e1e1a34342 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:38.914526 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de7504dc-c2b3-4672-8731-07e1e1a34342 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:38.914918 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-de7504dc-c2b3-4672-8731-07e1e1a34342 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:39.078219 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-de7504dc-c2b3-4672-8731-07e1e1a34342 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:39.087265 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-de7504dc-c2b3-4672-8731-07e1e1a34342 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.190496 Oct 20 09:09:39.087852 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 569/1138] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:38 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:40.111428 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f9108e2c-4f0b-4412-82a9-c21fc05337c3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:40.112127 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f9108e2c-4f0b-4412-82a9-c21fc05337c3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:40.118250 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9108e2c-4f0b-4412-82a9-c21fc05337c3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:40.118766 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9108e2c-4f0b-4412-82a9-c21fc05337c3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:40.119203 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9108e2c-4f0b-4412-82a9-c21fc05337c3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:40.119890 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9108e2c-4f0b-4412-82a9-c21fc05337c3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:40.120344 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9108e2c-4f0b-4412-82a9-c21fc05337c3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:40.120772 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9108e2c-4f0b-4412-82a9-c21fc05337c3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:40.254012 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f9108e2c-4f0b-4412-82a9-c21fc05337c3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:40.261645 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f9108e2c-4f0b-4412-82a9-c21fc05337c3 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.155458 Oct 20 09:09:40.262218 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 570/1139] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:40 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:41.284979 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-088b1866-45a2-4bf1-8ba2-910b42e97d2f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:41.285587 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-088b1866-45a2-4bf1-8ba2-910b42e97d2f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:41.292045 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-088b1866-45a2-4bf1-8ba2-910b42e97d2f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:41.292526 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-088b1866-45a2-4bf1-8ba2-910b42e97d2f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:41.292962 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-088b1866-45a2-4bf1-8ba2-910b42e97d2f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:41.293649 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-088b1866-45a2-4bf1-8ba2-910b42e97d2f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:41.294100 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-088b1866-45a2-4bf1-8ba2-910b42e97d2f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:41.294551 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-088b1866-45a2-4bf1-8ba2-910b42e97d2f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:41.421569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-088b1866-45a2-4bf1-8ba2-910b42e97d2f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:41.430315 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-088b1866-45a2-4bf1-8ba2-910b42e97d2f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.149648 Oct 20 09:09:41.430902 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 570/1140] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:41 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:42.452852 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-bfb480a3-754c-4bf3-ad71-38ed487657a4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:42.453457 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-bfb480a3-754c-4bf3-ad71-38ed487657a4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:42.459863 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfb480a3-754c-4bf3-ad71-38ed487657a4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:42.460338 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfb480a3-754c-4bf3-ad71-38ed487657a4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:42.460769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfb480a3-754c-4bf3-ad71-38ed487657a4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:42.461450 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfb480a3-754c-4bf3-ad71-38ed487657a4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:42.461900 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfb480a3-754c-4bf3-ad71-38ed487657a4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:42.462316 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-bfb480a3-754c-4bf3-ad71-38ed487657a4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:42.603143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-bfb480a3-754c-4bf3-ad71-38ed487657a4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:42.611501 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-bfb480a3-754c-4bf3-ad71-38ed487657a4 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1267 microversion: 2.1 time: 0.162522 Oct 20 09:09:42.612303 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 571/1141] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:42 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1267 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:43.635281 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f4c61e26-cf87-4015-b824-76b0ef0f859c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:43.635883 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f4c61e26-cf87-4015-b824-76b0ef0f859c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:43.643131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f4c61e26-cf87-4015-b824-76b0ef0f859c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:43.643625 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f4c61e26-cf87-4015-b824-76b0ef0f859c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:43.644064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f4c61e26-cf87-4015-b824-76b0ef0f859c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:43.644766 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f4c61e26-cf87-4015-b824-76b0ef0f859c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:43.645221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f4c61e26-cf87-4015-b824-76b0ef0f859c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:43.645648 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f4c61e26-cf87-4015-b824-76b0ef0f859c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:44.107021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f4c61e26-cf87-4015-b824-76b0ef0f859c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:44.110976 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=100 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:09:44.115998 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f4c61e26-cf87-4015-b824-76b0ef0f859c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1455 microversion: 2.1 time: 0.484658 Oct 20 09:09:44.116572 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 571/1142] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:43 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1455 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:44.138971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-6ba64571-3f16-4768-b2f4-7f45fd97ec25 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:09:44.140542 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-6ba64571-3f16-4768-b2f4-7f45fd97ec25 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:09:44.147347 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ba64571-3f16-4768-b2f4-7f45fd97ec25 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:44.147836 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ba64571-3f16-4768-b2f4-7f45fd97ec25 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:44.148272 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ba64571-3f16-4768-b2f4-7f45fd97ec25 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:44.148971 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ba64571-3f16-4768-b2f4-7f45fd97ec25 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:44.149424 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ba64571-3f16-4768-b2f4-7f45fd97ec25 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:44.149854 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-6ba64571-3f16-4768-b2f4-7f45fd97ec25 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:44.301814 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-6ba64571-3f16-4768-b2f4-7f45fd97ec25 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:09:44.310837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-6ba64571-3f16-4768-b2f4-7f45fd97ec25 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1455 microversion: 2.1 time: 0.175865 Oct 20 09:09:44.311421 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 572/1143] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:09:44 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1455 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:09:46.046297 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-abe2cda9-707d-412f-bd9a-be405334bdc9 req-6c185386-d490-44f7-b09d-8c048a08b7bb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "842be7a4-f260-42e7-b34e-5f4369630d36", "tag": "9d2af910-87ab-4f40-ac9b-a6a140aaca39"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:09:46.056516 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-abe2cda9-707d-412f-bd9a-be405334bdc9 req-6c185386-d490-44f7-b09d-8c048a08b7bb service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:09:46.056991 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-abe2cda9-707d-412f-bd9a-be405334bdc9 req-6c185386-d490-44f7-b09d-8c048a08b7bb service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:09:46.057486 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-abe2cda9-707d-412f-bd9a-be405334bdc9 req-6c185386-d490-44f7-b09d-8c048a08b7bb service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:09:46.071657 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-abe2cda9-707d-412f-bd9a-be405334bdc9 req-6c185386-d490-44f7-b09d-8c048a08b7bb service nova] Creating event network-changed:9d2af910-87ab-4f40-ac9b-a6a140aaca39 for instance 842be7a4-f260-42e7-b34e-5f4369630d36 on hv-cinder-maste Oct 20 09:09:46.076803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-abe2cda9-707d-412f-bd9a-be405334bdc9 req-6c185386-d490-44f7-b09d-8c048a08b7bb service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034159 Oct 20 09:09:46.077522 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 572/1144] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:09:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:10:34.702023 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-dd5580f1-67e0-4d7e-b65a-68ba1b47f2a3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:10:34.702627 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-dd5580f1-67e0-4d7e-b65a-68ba1b47f2a3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:34.705402 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=11 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:10:34.706044 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=64 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:10:34.711411 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dd5580f1-67e0-4d7e-b65a-68ba1b47f2a3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:34.711891 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dd5580f1-67e0-4d7e-b65a-68ba1b47f2a3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:34.712325 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dd5580f1-67e0-4d7e-b65a-68ba1b47f2a3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:34.713011 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dd5580f1-67e0-4d7e-b65a-68ba1b47f2a3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:34.713459 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dd5580f1-67e0-4d7e-b65a-68ba1b47f2a3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:34.713899 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-dd5580f1-67e0-4d7e-b65a-68ba1b47f2a3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:34.736864 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-dd5580f1-67e0-4d7e-b65a-68ba1b47f2a3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lazy-loading 'flavor' on Instance uuid c8a04d20-065b-47fd-b7e2-8238e34be187 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:10:34.809375 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-dd5580f1-67e0-4d7e-b65a-68ba1b47f2a3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "POST /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187/action" status: 202 len: 0 microversion: 2.1 time: 0.111288 Oct 20 09:10:34.809970 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 573/1145] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:10:34 2022] POST /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187/action => generated 0 bytes in 113 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:10:34.819269 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4d280b63-a03b-4c5a-b591-541345bd2361 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:34.820211 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4d280b63-a03b-4c5a-b591-541345bd2361 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:34.822917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=19 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:10:34.823554 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:10:34.831018 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4d280b63-a03b-4c5a-b591-541345bd2361 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:34.831774 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4d280b63-a03b-4c5a-b591-541345bd2361 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:34.832215 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4d280b63-a03b-4c5a-b591-541345bd2361 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:34.833076 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4d280b63-a03b-4c5a-b591-541345bd2361 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:34.833529 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4d280b63-a03b-4c5a-b591-541345bd2361 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:34.834208 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4d280b63-a03b-4c5a-b591-541345bd2361 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:35.136998 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4d280b63-a03b-4c5a-b591-541345bd2361 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:35.145098 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4d280b63-a03b-4c5a-b591-541345bd2361 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.329960 Oct 20 09:10:35.145667 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 573/1146] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:34 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:36.169930 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-26839aa6-520e-4aa0-b65e-c6414d0ad2e7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:36.170647 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-26839aa6-520e-4aa0-b65e-c6414d0ad2e7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:36.177417 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-26839aa6-520e-4aa0-b65e-c6414d0ad2e7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:36.177891 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-26839aa6-520e-4aa0-b65e-c6414d0ad2e7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:36.178322 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-26839aa6-520e-4aa0-b65e-c6414d0ad2e7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:36.179032 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-26839aa6-520e-4aa0-b65e-c6414d0ad2e7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:36.179491 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-26839aa6-520e-4aa0-b65e-c6414d0ad2e7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:36.179898 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-26839aa6-520e-4aa0-b65e-c6414d0ad2e7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:36.366643 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-26839aa6-520e-4aa0-b65e-c6414d0ad2e7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:36.374642 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-26839aa6-520e-4aa0-b65e-c6414d0ad2e7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.209480 Oct 20 09:10:36.375208 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 574/1147] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:36 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:37.397796 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-91e56d93-3e4a-4841-ae54-81097e6f92ce tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:37.398404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-91e56d93-3e4a-4841-ae54-81097e6f92ce tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:37.404974 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-91e56d93-3e4a-4841-ae54-81097e6f92ce tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:37.405453 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-91e56d93-3e4a-4841-ae54-81097e6f92ce tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:37.405890 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-91e56d93-3e4a-4841-ae54-81097e6f92ce tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:37.406603 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-91e56d93-3e4a-4841-ae54-81097e6f92ce tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:37.407061 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-91e56d93-3e4a-4841-ae54-81097e6f92ce tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:37.407493 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-91e56d93-3e4a-4841-ae54-81097e6f92ce tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:37.535944 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-91e56d93-3e4a-4841-ae54-81097e6f92ce tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:37.543658 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-91e56d93-3e4a-4841-ae54-81097e6f92ce tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.149772 Oct 20 09:10:37.544220 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 574/1148] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:37 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:10:38.567090 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-be339884-7918-443b-9efc-d7d1f4471dde tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:38.567692 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-be339884-7918-443b-9efc-d7d1f4471dde tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:38.574051 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be339884-7918-443b-9efc-d7d1f4471dde tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:38.574555 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be339884-7918-443b-9efc-d7d1f4471dde tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:38.574982 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be339884-7918-443b-9efc-d7d1f4471dde tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:38.575650 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be339884-7918-443b-9efc-d7d1f4471dde tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:38.576091 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be339884-7918-443b-9efc-d7d1f4471dde tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:38.576503 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-be339884-7918-443b-9efc-d7d1f4471dde tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:38.705358 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-be339884-7918-443b-9efc-d7d1f4471dde tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:38.713066 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-be339884-7918-443b-9efc-d7d1f4471dde tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.149876 Oct 20 09:10:38.713640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 575/1149] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:38 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:10:39.737392 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a38ace7d-8134-4121-8f8e-e9f205bedb75 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:39.737981 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a38ace7d-8134-4121-8f8e-e9f205bedb75 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:39.744353 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a38ace7d-8134-4121-8f8e-e9f205bedb75 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:39.744838 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a38ace7d-8134-4121-8f8e-e9f205bedb75 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:39.745276 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a38ace7d-8134-4121-8f8e-e9f205bedb75 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:39.745993 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a38ace7d-8134-4121-8f8e-e9f205bedb75 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:39.746449 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a38ace7d-8134-4121-8f8e-e9f205bedb75 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:39.746898 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a38ace7d-8134-4121-8f8e-e9f205bedb75 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:39.874832 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a38ace7d-8134-4121-8f8e-e9f205bedb75 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:39.883108 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a38ace7d-8134-4121-8f8e-e9f205bedb75 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.150192 Oct 20 09:10:39.883674 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 575/1150] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:39 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:40.906372 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ede9bdc0-499c-4a0f-8974-2b502c79a2a1 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:40.907053 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ede9bdc0-499c-4a0f-8974-2b502c79a2a1 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:40.914013 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ede9bdc0-499c-4a0f-8974-2b502c79a2a1 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:40.914520 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ede9bdc0-499c-4a0f-8974-2b502c79a2a1 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:40.914961 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ede9bdc0-499c-4a0f-8974-2b502c79a2a1 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:40.915647 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ede9bdc0-499c-4a0f-8974-2b502c79a2a1 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:40.916101 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ede9bdc0-499c-4a0f-8974-2b502c79a2a1 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:40.916525 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ede9bdc0-499c-4a0f-8974-2b502c79a2a1 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:41.043799 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ede9bdc0-499c-4a0f-8974-2b502c79a2a1 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:41.051506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ede9bdc0-499c-4a0f-8974-2b502c79a2a1 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.149007 Oct 20 09:10:41.052077 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 576/1151] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:40 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:42.074383 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-b042f9a4-505e-4d0f-9c47-6994e3884336 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:42.075020 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-b042f9a4-505e-4d0f-9c47-6994e3884336 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:42.081923 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b042f9a4-505e-4d0f-9c47-6994e3884336 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:42.082403 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b042f9a4-505e-4d0f-9c47-6994e3884336 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:42.082876 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b042f9a4-505e-4d0f-9c47-6994e3884336 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:42.083565 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b042f9a4-505e-4d0f-9c47-6994e3884336 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:42.084040 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b042f9a4-505e-4d0f-9c47-6994e3884336 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:42.084465 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-b042f9a4-505e-4d0f-9c47-6994e3884336 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:42.216569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-b042f9a4-505e-4d0f-9c47-6994e3884336 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:42.224357 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-b042f9a4-505e-4d0f-9c47-6994e3884336 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.153860 Oct 20 09:10:42.224919 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 576/1152] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:42 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:43.247201 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f9cbcc3b-2a9a-4bf5-b159-6d6dcfdfca4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:43.247791 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f9cbcc3b-2a9a-4bf5-b159-6d6dcfdfca4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:43.254204 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9cbcc3b-2a9a-4bf5-b159-6d6dcfdfca4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:43.254725 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9cbcc3b-2a9a-4bf5-b159-6d6dcfdfca4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:43.255165 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9cbcc3b-2a9a-4bf5-b159-6d6dcfdfca4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:43.255849 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9cbcc3b-2a9a-4bf5-b159-6d6dcfdfca4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:43.256304 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9cbcc3b-2a9a-4bf5-b159-6d6dcfdfca4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:43.256725 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f9cbcc3b-2a9a-4bf5-b159-6d6dcfdfca4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:43.386459 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f9cbcc3b-2a9a-4bf5-b159-6d6dcfdfca4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:43.394727 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f9cbcc3b-2a9a-4bf5-b159-6d6dcfdfca4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.151681 Oct 20 09:10:43.395296 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 577/1153] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:43 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:44.418372 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a272afc6-6e5c-4e6d-9e9b-8b1913c8badc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:44.419026 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a272afc6-6e5c-4e6d-9e9b-8b1913c8badc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:44.426082 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a272afc6-6e5c-4e6d-9e9b-8b1913c8badc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:44.426644 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a272afc6-6e5c-4e6d-9e9b-8b1913c8badc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:44.427075 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a272afc6-6e5c-4e6d-9e9b-8b1913c8badc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:44.427754 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a272afc6-6e5c-4e6d-9e9b-8b1913c8badc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:44.428201 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a272afc6-6e5c-4e6d-9e9b-8b1913c8badc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:44.428619 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a272afc6-6e5c-4e6d-9e9b-8b1913c8badc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:44.555627 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a272afc6-6e5c-4e6d-9e9b-8b1913c8badc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:44.563297 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a272afc6-6e5c-4e6d-9e9b-8b1913c8badc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.148773 Oct 20 09:10:44.563902 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 577/1154] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:44 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:45.587016 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-534a833f-0cfc-4387-9eb2-4fa60a5646f0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:45.587626 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-534a833f-0cfc-4387-9eb2-4fa60a5646f0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:45.594390 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-534a833f-0cfc-4387-9eb2-4fa60a5646f0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:45.595547 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-534a833f-0cfc-4387-9eb2-4fa60a5646f0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:45.595978 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-534a833f-0cfc-4387-9eb2-4fa60a5646f0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:45.596657 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-534a833f-0cfc-4387-9eb2-4fa60a5646f0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:45.597102 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-534a833f-0cfc-4387-9eb2-4fa60a5646f0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:45.597520 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-534a833f-0cfc-4387-9eb2-4fa60a5646f0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:45.724079 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-534a833f-0cfc-4387-9eb2-4fa60a5646f0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:45.731832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-534a833f-0cfc-4387-9eb2-4fa60a5646f0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.148706 Oct 20 09:10:45.732405 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 578/1155] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:45 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:46.755131 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-ae0adbdf-ac21-4fc9-a962-1c95132c7368 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:46.755740 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-ae0adbdf-ac21-4fc9-a962-1c95132c7368 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:46.762363 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae0adbdf-ac21-4fc9-a962-1c95132c7368 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:46.762880 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae0adbdf-ac21-4fc9-a962-1c95132c7368 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:46.763316 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae0adbdf-ac21-4fc9-a962-1c95132c7368 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:46.764014 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae0adbdf-ac21-4fc9-a962-1c95132c7368 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:46.764463 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae0adbdf-ac21-4fc9-a962-1c95132c7368 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:46.764885 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-ae0adbdf-ac21-4fc9-a962-1c95132c7368 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:46.896806 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-ae0adbdf-ac21-4fc9-a962-1c95132c7368 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:46.904506 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-ae0adbdf-ac21-4fc9-a962-1c95132c7368 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.153750 Oct 20 09:10:46.905068 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 578/1156] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:46 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:47.927951 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a0a8cbe7-7f1d-47f2-8484-3980e56be0d5 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:47.928569 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a0a8cbe7-7f1d-47f2-8484-3980e56be0d5 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:47.931619 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 20 09:10:47.931753 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Thu Oct 20 09:10:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 (ip 10.222.0.36) !!! Oct 20 09:10:47.938920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a0a8cbe7-7f1d-47f2-8484-3980e56be0d5 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:47.939402 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a0a8cbe7-7f1d-47f2-8484-3980e56be0d5 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:47.939832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a0a8cbe7-7f1d-47f2-8484-3980e56be0d5 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:47.940517 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a0a8cbe7-7f1d-47f2-8484-3980e56be0d5 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:47.940963 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a0a8cbe7-7f1d-47f2-8484-3980e56be0d5 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:47.941381 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a0a8cbe7-7f1d-47f2-8484-3980e56be0d5 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:48.325502 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a0a8cbe7-7f1d-47f2-8484-3980e56be0d5 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:48.333248 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a0a8cbe7-7f1d-47f2-8484-3980e56be0d5 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.409158 Oct 20 09:10:48.333822 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 579/1157] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:47 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:49.356502 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-4f0f8949-a38d-45fd-9872-774a5cdad018 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:49.357133 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-4f0f8949-a38d-45fd-9872-774a5cdad018 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:49.364471 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f0f8949-a38d-45fd-9872-774a5cdad018 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:49.364947 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f0f8949-a38d-45fd-9872-774a5cdad018 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:49.365379 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f0f8949-a38d-45fd-9872-774a5cdad018 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:49.366064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f0f8949-a38d-45fd-9872-774a5cdad018 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:49.366529 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f0f8949-a38d-45fd-9872-774a5cdad018 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:49.366949 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-4f0f8949-a38d-45fd-9872-774a5cdad018 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:49.496726 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-4f0f8949-a38d-45fd-9872-774a5cdad018 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:49.504292 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-4f0f8949-a38d-45fd-9872-774a5cdad018 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1506 microversion: 2.1 time: 0.151694 Oct 20 09:10:49.504885 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 579/1158] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:49 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1506 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:49.526385 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7f868c66-9f74-4e3d-97f8-1a87a4de0e15 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:10:49.527015 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7f868c66-9f74-4e3d-97f8-1a87a4de0e15 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:49.533832 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f868c66-9f74-4e3d-97f8-1a87a4de0e15 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:49.534309 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f868c66-9f74-4e3d-97f8-1a87a4de0e15 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:49.534779 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f868c66-9f74-4e3d-97f8-1a87a4de0e15 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:49.535460 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f868c66-9f74-4e3d-97f8-1a87a4de0e15 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:49.535924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f868c66-9f74-4e3d-97f8-1a87a4de0e15 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:49.536339 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7f868c66-9f74-4e3d-97f8-1a87a4de0e15 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:49.567662 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7f868c66-9f74-4e3d-97f8-1a87a4de0e15 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=119067) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 20 09:10:49.634659 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7f868c66-9f74-4e3d-97f8-1a87a4de0e15 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "POST /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187/action" status: 202 len: 0 microversion: 2.1 time: 0.112099 Oct 20 09:10:49.635179 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 580/1159] 10.222.0.36 () {60 vars in 1278 bytes} [Thu Oct 20 09:10:49 2022] POST /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187/action => generated 0 bytes in 113 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:10:49.643162 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8d9fbfb2-c304-4647-8686-f9c58d198fae tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:49.643794 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8d9fbfb2-c304-4647-8686-f9c58d198fae tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:49.649600 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d9fbfb2-c304-4647-8686-f9c58d198fae tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:49.650075 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d9fbfb2-c304-4647-8686-f9c58d198fae tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:49.650534 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d9fbfb2-c304-4647-8686-f9c58d198fae tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:49.651215 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d9fbfb2-c304-4647-8686-f9c58d198fae tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:49.651660 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d9fbfb2-c304-4647-8686-f9c58d198fae tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:49.652085 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8d9fbfb2-c304-4647-8686-f9c58d198fae tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:49.820721 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-8d9fbfb2-c304-4647-8686-f9c58d198fae tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:49.828535 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8d9fbfb2-c304-4647-8686-f9c58d198fae tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1514 microversion: 2.1 time: 0.189222 Oct 20 09:10:49.829700 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 580/1160] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:49 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1514 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:50.853730 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-4afeeb86-ee49-4b3b-913a-df0aab6a70bf tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:50.854418 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-4afeeb86-ee49-4b3b-913a-df0aab6a70bf tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:50.860994 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4afeeb86-ee49-4b3b-913a-df0aab6a70bf tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:50.861473 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4afeeb86-ee49-4b3b-913a-df0aab6a70bf tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:50.861903 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4afeeb86-ee49-4b3b-913a-df0aab6a70bf tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:50.862623 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4afeeb86-ee49-4b3b-913a-df0aab6a70bf tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:50.863071 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4afeeb86-ee49-4b3b-913a-df0aab6a70bf tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:50.863490 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-4afeeb86-ee49-4b3b-913a-df0aab6a70bf tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:50.959755 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-da17416d-8807-4063-909e-55548b1672b9 req-76a74246-3793-4b01-ba46-0d63c80c080e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c8a04d20-065b-47fd-b7e2-8238e34be187", "name": "network-vif-unplugged", "status": "completed", "tag": "731aa0eb-33e7-4deb-b323-29e7908786e9"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:10:50.972105 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-da17416d-8807-4063-909e-55548b1672b9 req-76a74246-3793-4b01-ba46-0d63c80c080e service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:50.972855 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-da17416d-8807-4063-909e-55548b1672b9 req-76a74246-3793-4b01-ba46-0d63c80c080e service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:50.973278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-da17416d-8807-4063-909e-55548b1672b9 req-76a74246-3793-4b01-ba46-0d63c80c080e service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:50.989459 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.compute.server_external_events [req-da17416d-8807-4063-909e-55548b1672b9 req-76a74246-3793-4b01-ba46-0d63c80c080e service nova] Unable to find a host for instance c8a04d20-065b-47fd-b7e2-8238e34be187. Dropping event network-vif-unplugged {{(pid=119068) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 20 09:10:50.990331 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-da17416d-8807-4063-909e-55548b1672b9 req-76a74246-3793-4b01-ba46-0d63c80c080e service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.034733 Oct 20 09:10:50.991157 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 581/1161] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:10:50 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 36 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Oct 20 09:10:51.001891 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-4afeeb86-ee49-4b3b-913a-df0aab6a70bf tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:51.009693 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-4afeeb86-ee49-4b3b-913a-df0aab6a70bf tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1516 microversion: 2.1 time: 0.160540 Oct 20 09:10:51.010722 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 581/1162] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:50 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1516 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:52.038055 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-652ed97c-a8dc-4e1a-94b7-9c34dfb72eb0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:52.038699 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-652ed97c-a8dc-4e1a-94b7-9c34dfb72eb0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:52.045026 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-652ed97c-a8dc-4e1a-94b7-9c34dfb72eb0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:52.045510 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-652ed97c-a8dc-4e1a-94b7-9c34dfb72eb0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:52.045953 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-652ed97c-a8dc-4e1a-94b7-9c34dfb72eb0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:52.046683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-652ed97c-a8dc-4e1a-94b7-9c34dfb72eb0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:52.047138 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-652ed97c-a8dc-4e1a-94b7-9c34dfb72eb0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:52.047557 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-652ed97c-a8dc-4e1a-94b7-9c34dfb72eb0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:52.242969 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-652ed97c-a8dc-4e1a-94b7-9c34dfb72eb0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:52.250642 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-652ed97c-a8dc-4e1a-94b7-9c34dfb72eb0 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.216439 Oct 20 09:10:52.251217 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 582/1163] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:52 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:53.274691 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0980a0b1-2b56-4347-aa33-efc529b6da61 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:53.275310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0980a0b1-2b56-4347-aa33-efc529b6da61 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:53.281724 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0980a0b1-2b56-4347-aa33-efc529b6da61 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:53.282734 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0980a0b1-2b56-4347-aa33-efc529b6da61 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:53.283167 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0980a0b1-2b56-4347-aa33-efc529b6da61 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:53.283848 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0980a0b1-2b56-4347-aa33-efc529b6da61 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:53.284298 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0980a0b1-2b56-4347-aa33-efc529b6da61 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:53.284720 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0980a0b1-2b56-4347-aa33-efc529b6da61 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:53.414357 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0980a0b1-2b56-4347-aa33-efc529b6da61 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:53.422061 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0980a0b1-2b56-4347-aa33-efc529b6da61 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.151304 Oct 20 09:10:53.423188 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 582/1164] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:53 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:54.444496 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c10bb229-3127-40c0-8bd0-6670da07b5bc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:54.445129 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c10bb229-3127-40c0-8bd0-6670da07b5bc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:54.451492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c10bb229-3127-40c0-8bd0-6670da07b5bc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:54.451970 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c10bb229-3127-40c0-8bd0-6670da07b5bc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:54.452404 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c10bb229-3127-40c0-8bd0-6670da07b5bc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:54.453084 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c10bb229-3127-40c0-8bd0-6670da07b5bc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:54.453531 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c10bb229-3127-40c0-8bd0-6670da07b5bc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:54.454492 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c10bb229-3127-40c0-8bd0-6670da07b5bc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:54.580114 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c10bb229-3127-40c0-8bd0-6670da07b5bc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:54.587759 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c10bb229-3127-40c0-8bd0-6670da07b5bc tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.147174 Oct 20 09:10:54.588317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 583/1165] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:54 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:55.612257 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-a9596ac7-492f-43ed-a394-bad352ea8f96 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:55.612865 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-a9596ac7-492f-43ed-a394-bad352ea8f96 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:55.619444 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a9596ac7-492f-43ed-a394-bad352ea8f96 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:55.619921 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a9596ac7-492f-43ed-a394-bad352ea8f96 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:55.620354 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a9596ac7-492f-43ed-a394-bad352ea8f96 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:55.621038 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a9596ac7-492f-43ed-a394-bad352ea8f96 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:55.621491 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a9596ac7-492f-43ed-a394-bad352ea8f96 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:55.621915 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-a9596ac7-492f-43ed-a394-bad352ea8f96 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:55.748004 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-a9596ac7-492f-43ed-a394-bad352ea8f96 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:55.755705 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-a9596ac7-492f-43ed-a394-bad352ea8f96 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.147963 Oct 20 09:10:55.756261 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 583/1166] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:55 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:56.778259 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-c6b27577-c2a9-4d68-8b96-0b117ca51f74 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:56.778916 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-c6b27577-c2a9-4d68-8b96-0b117ca51f74 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:56.786890 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6b27577-c2a9-4d68-8b96-0b117ca51f74 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:56.787364 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6b27577-c2a9-4d68-8b96-0b117ca51f74 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:56.787795 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6b27577-c2a9-4d68-8b96-0b117ca51f74 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:56.788473 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6b27577-c2a9-4d68-8b96-0b117ca51f74 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:56.788919 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6b27577-c2a9-4d68-8b96-0b117ca51f74 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:56.789336 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-c6b27577-c2a9-4d68-8b96-0b117ca51f74 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:56.915513 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-c6b27577-c2a9-4d68-8b96-0b117ca51f74 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:56.923141 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-c6b27577-c2a9-4d68-8b96-0b117ca51f74 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.148793 Oct 20 09:10:56.923696 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 584/1167] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:56 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:57.946487 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-93cd2739-25ea-4502-b552-bb5fc9e67000 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:57.947107 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-93cd2739-25ea-4502-b552-bb5fc9e67000 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:57.953235 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93cd2739-25ea-4502-b552-bb5fc9e67000 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:57.953710 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93cd2739-25ea-4502-b552-bb5fc9e67000 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:57.954143 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93cd2739-25ea-4502-b552-bb5fc9e67000 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:57.954845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93cd2739-25ea-4502-b552-bb5fc9e67000 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:57.955301 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93cd2739-25ea-4502-b552-bb5fc9e67000 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:57.955718 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-93cd2739-25ea-4502-b552-bb5fc9e67000 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:58.085649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-93cd2739-25ea-4502-b552-bb5fc9e67000 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:58.093655 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-93cd2739-25ea-4502-b552-bb5fc9e67000 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.151100 Oct 20 09:10:58.094227 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 584/1168] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:57 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:10:59.116911 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-56102681-4e46-4b88-9d04-0987e1f3fc4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:10:59.117521 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-56102681-4e46-4b88-9d04-0987e1f3fc4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:10:59.123942 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56102681-4e46-4b88-9d04-0987e1f3fc4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:59.124423 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56102681-4e46-4b88-9d04-0987e1f3fc4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:59.124858 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56102681-4e46-4b88-9d04-0987e1f3fc4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:59.126168 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56102681-4e46-4b88-9d04-0987e1f3fc4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:10:59.126654 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56102681-4e46-4b88-9d04-0987e1f3fc4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:10:59.127079 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-56102681-4e46-4b88-9d04-0987e1f3fc4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:10:59.258700 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-56102681-4e46-4b88-9d04-0987e1f3fc4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:10:59.266376 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-56102681-4e46-4b88-9d04-0987e1f3fc4a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.153369 Oct 20 09:10:59.266981 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 585/1169] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:10:59 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:00.288974 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-b0ac04e4-807b-4a19-8a3e-20a4aab72025 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:00.289580 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-b0ac04e4-807b-4a19-8a3e-20a4aab72025 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:00.296144 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0ac04e4-807b-4a19-8a3e-20a4aab72025 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:00.296617 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0ac04e4-807b-4a19-8a3e-20a4aab72025 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:00.297044 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0ac04e4-807b-4a19-8a3e-20a4aab72025 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:00.298373 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0ac04e4-807b-4a19-8a3e-20a4aab72025 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:00.298853 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0ac04e4-807b-4a19-8a3e-20a4aab72025 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:00.299271 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-b0ac04e4-807b-4a19-8a3e-20a4aab72025 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:00.426912 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-b0ac04e4-807b-4a19-8a3e-20a4aab72025 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:00.434694 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-b0ac04e4-807b-4a19-8a3e-20a4aab72025 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.149472 Oct 20 09:11:00.435252 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 585/1170] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:00 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:01.462096 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-f2f4c0d0-2783-4e99-8a2d-8e86e7958540 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:01.462775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-f2f4c0d0-2783-4e99-8a2d-8e86e7958540 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:01.469132 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2f4c0d0-2783-4e99-8a2d-8e86e7958540 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:01.469619 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2f4c0d0-2783-4e99-8a2d-8e86e7958540 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:01.470060 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2f4c0d0-2783-4e99-8a2d-8e86e7958540 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:01.470770 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2f4c0d0-2783-4e99-8a2d-8e86e7958540 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:01.471221 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2f4c0d0-2783-4e99-8a2d-8e86e7958540 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:01.471645 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-f2f4c0d0-2783-4e99-8a2d-8e86e7958540 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:01.600818 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-f2f4c0d0-2783-4e99-8a2d-8e86e7958540 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:01.608546 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-f2f4c0d0-2783-4e99-8a2d-8e86e7958540 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.150309 Oct 20 09:11:01.609118 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 586/1171] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:01 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:02.632545 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-f3462577-9dfd-4cc9-8277-231e0dd2f37d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:02.633088 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-f3462577-9dfd-4cc9-8277-231e0dd2f37d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:02.639533 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3462577-9dfd-4cc9-8277-231e0dd2f37d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:02.640009 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3462577-9dfd-4cc9-8277-231e0dd2f37d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:02.640441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3462577-9dfd-4cc9-8277-231e0dd2f37d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:02.641120 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3462577-9dfd-4cc9-8277-231e0dd2f37d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:02.641569 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3462577-9dfd-4cc9-8277-231e0dd2f37d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:02.641989 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-f3462577-9dfd-4cc9-8277-231e0dd2f37d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:02.659474 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:11:02.774707 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-f3462577-9dfd-4cc9-8277-231e0dd2f37d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:02.782310 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-f3462577-9dfd-4cc9-8277-231e0dd2f37d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.153719 Oct 20 09:11:02.782902 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 586/1172] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:02 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:03.057602 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-7f0d50a9-a3b8-43bc-ac4e-26c391e3bf69 req-ee9f5620-f622-4dc5-94ac-4e35d6637bde service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c8a04d20-065b-47fd-b7e2-8238e34be187", "name": "network-vif-plugged", "status": "completed", "tag": "731aa0eb-33e7-4deb-b323-29e7908786e9"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:11:03.068145 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-7f0d50a9-a3b8-43bc-ac4e-26c391e3bf69 req-ee9f5620-f622-4dc5-94ac-4e35d6637bde service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:03.069250 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-7f0d50a9-a3b8-43bc-ac4e-26c391e3bf69 req-ee9f5620-f622-4dc5-94ac-4e35d6637bde service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:03.069814 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-7f0d50a9-a3b8-43bc-ac4e-26c391e3bf69 req-ee9f5620-f622-4dc5-94ac-4e35d6637bde service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:03.083523 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-7f0d50a9-a3b8-43bc-ac4e-26c391e3bf69 req-ee9f5620-f622-4dc5-94ac-4e35d6637bde service nova] Creating event network-vif-plugged:731aa0eb-33e7-4deb-b323-29e7908786e9 for instance c8a04d20-065b-47fd-b7e2-8238e34be187 on hv-cinder-maste Oct 20 09:11:03.088592 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-7f0d50a9-a3b8-43bc-ac4e-26c391e3bf69 req-ee9f5620-f622-4dc5-94ac-4e35d6637bde service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034921 Oct 20 09:11:03.089183 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 587/1173] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:11:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:11:03.805133 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-7c445692-ced1-4703-bd35-1d8d815a5e4d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:03.805654 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-7c445692-ced1-4703-bd35-1d8d815a5e4d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:03.811993 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7c445692-ced1-4703-bd35-1d8d815a5e4d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:03.812474 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7c445692-ced1-4703-bd35-1d8d815a5e4d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:03.812907 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7c445692-ced1-4703-bd35-1d8d815a5e4d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:03.814096 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7c445692-ced1-4703-bd35-1d8d815a5e4d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:03.814566 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7c445692-ced1-4703-bd35-1d8d815a5e4d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:03.814991 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-7c445692-ced1-4703-bd35-1d8d815a5e4d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:03.942057 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-7c445692-ced1-4703-bd35-1d8d815a5e4d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:03.949835 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-7c445692-ced1-4703-bd35-1d8d815a5e4d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.148615 Oct 20 09:11:03.950405 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 587/1174] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:03 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:04.973959 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-84b2f4c5-c08d-4791-990f-4aadcaad29e2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:04.974731 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-84b2f4c5-c08d-4791-990f-4aadcaad29e2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:04.977190 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=31 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:11:04.982898 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-84b2f4c5-c08d-4791-990f-4aadcaad29e2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:04.983385 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-84b2f4c5-c08d-4791-990f-4aadcaad29e2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:04.983823 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-84b2f4c5-c08d-4791-990f-4aadcaad29e2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:04.984509 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-84b2f4c5-c08d-4791-990f-4aadcaad29e2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:04.984964 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-84b2f4c5-c08d-4791-990f-4aadcaad29e2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:04.985391 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-84b2f4c5-c08d-4791-990f-4aadcaad29e2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:04.989495 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=83 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:11:05.123596 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-84b2f4c5-c08d-4791-990f-4aadcaad29e2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:05.131320 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-84b2f4c5-c08d-4791-990f-4aadcaad29e2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1572 microversion: 2.1 time: 0.162227 Oct 20 09:11:05.131882 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 588/1175] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:04 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1572 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:06.154441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-0901843b-04a4-4ede-8443-548a7027f2c4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:06.155071 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-0901843b-04a4-4ede-8443-548a7027f2c4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:06.157991 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:11:06.163165 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0901843b-04a4-4ede-8443-548a7027f2c4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:06.163648 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0901843b-04a4-4ede-8443-548a7027f2c4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:06.164080 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0901843b-04a4-4ede-8443-548a7027f2c4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:06.164762 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0901843b-04a4-4ede-8443-548a7027f2c4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:06.165206 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0901843b-04a4-4ede-8443-548a7027f2c4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:06.165629 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-0901843b-04a4-4ede-8443-548a7027f2c4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:06.293546 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-0901843b-04a4-4ede-8443-548a7027f2c4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:06.301205 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-0901843b-04a4-4ede-8443-548a7027f2c4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1559 microversion: 2.1 time: 0.150568 Oct 20 09:11:06.301771 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 588/1176] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:06 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:26.225569 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8937d640-dcba-451d-9b27-24cb16f22117 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:26.226352 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8937d640-dcba-451d-9b27-24cb16f22117 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:26.228829 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:11:26.229264 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=1 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:11:26.229721 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=14 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:11:26.230322 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=5 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:11:26.236660 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8937d640-dcba-451d-9b27-24cb16f22117 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:26.237289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8937d640-dcba-451d-9b27-24cb16f22117 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:26.237813 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8937d640-dcba-451d-9b27-24cb16f22117 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:26.238539 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8937d640-dcba-451d-9b27-24cb16f22117 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:26.238998 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8937d640-dcba-451d-9b27-24cb16f22117 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:26.239424 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8937d640-dcba-451d-9b27-24cb16f22117 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:26.266633 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8937d640-dcba-451d-9b27-24cb16f22117 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:11:26.288951 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-69eb9aba-1229-4083-842b-4d26e59de7f1 req-754402ee-d3ae-4a0a-89ce-44a2d144bccc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "842be7a4-f260-42e7-b34e-5f4369630d36", "tag": "9d2af910-87ab-4f40-ac9b-a6a140aaca39"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:11:26.299184 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-69eb9aba-1229-4083-842b-4d26e59de7f1 req-754402ee-d3ae-4a0a-89ce-44a2d144bccc service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:26.299743 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-69eb9aba-1229-4083-842b-4d26e59de7f1 req-754402ee-d3ae-4a0a-89ce-44a2d144bccc service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:26.300228 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-69eb9aba-1229-4083-842b-4d26e59de7f1 req-754402ee-d3ae-4a0a-89ce-44a2d144bccc service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:26.312345 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-69eb9aba-1229-4083-842b-4d26e59de7f1 req-754402ee-d3ae-4a0a-89ce-44a2d144bccc service nova] Creating event network-changed:9d2af910-87ab-4f40-ac9b-a6a140aaca39 for instance 842be7a4-f260-42e7-b34e-5f4369630d36 on hv-cinder-maste Oct 20 09:11:26.317665 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-69eb9aba-1229-4083-842b-4d26e59de7f1 req-754402ee-d3ae-4a0a-89ce-44a2d144bccc service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.093793 Oct 20 09:11:26.318871 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 589/1177] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:11:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:11:26.349723 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8937d640-dcba-451d-9b27-24cb16f22117 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "DELETE /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 204 len: 0 microversion: 2.1 time: 0.127887 Oct 20 09:11:26.350407 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 589/1178] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:11:26 2022] DELETE /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:11:26.359151 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-77f93e6a-fa1d-4b2d-9499-2a724984a2ca tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:26.360740 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-77f93e6a-fa1d-4b2d-9499-2a724984a2ca tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:26.367168 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77f93e6a-fa1d-4b2d-9499-2a724984a2ca tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:26.367895 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77f93e6a-fa1d-4b2d-9499-2a724984a2ca tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:26.368506 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77f93e6a-fa1d-4b2d-9499-2a724984a2ca tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:26.369374 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77f93e6a-fa1d-4b2d-9499-2a724984a2ca tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:26.369986 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77f93e6a-fa1d-4b2d-9499-2a724984a2ca tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:26.370627 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-77f93e6a-fa1d-4b2d-9499-2a724984a2ca tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:26.506702 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-77f93e6a-fa1d-4b2d-9499-2a724984a2ca tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:26.514668 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-77f93e6a-fa1d-4b2d-9499-2a724984a2ca tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1579 microversion: 2.1 time: 0.159494 Oct 20 09:11:26.515139 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 590/1179] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:26 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1579 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:27.537786 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-21fe27d6-dee1-47a4-b79c-0a0dbb71fef1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:27.538410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-21fe27d6-dee1-47a4-b79c-0a0dbb71fef1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:27.544749 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21fe27d6-dee1-47a4-b79c-0a0dbb71fef1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:27.545227 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21fe27d6-dee1-47a4-b79c-0a0dbb71fef1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:27.545665 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21fe27d6-dee1-47a4-b79c-0a0dbb71fef1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:27.546356 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21fe27d6-dee1-47a4-b79c-0a0dbb71fef1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:27.547349 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21fe27d6-dee1-47a4-b79c-0a0dbb71fef1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:27.547775 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-21fe27d6-dee1-47a4-b79c-0a0dbb71fef1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:27.679137 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-21fe27d6-dee1-47a4-b79c-0a0dbb71fef1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:27.686892 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-21fe27d6-dee1-47a4-b79c-0a0dbb71fef1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1579 microversion: 2.1 time: 0.153427 Oct 20 09:11:27.687480 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 590/1180] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:27 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1579 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:28.709684 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-75c357bb-48c1-4417-8fe6-bfd3ff295bf1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:28.710286 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-75c357bb-48c1-4417-8fe6-bfd3ff295bf1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:28.716928 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c357bb-48c1-4417-8fe6-bfd3ff295bf1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:28.717405 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c357bb-48c1-4417-8fe6-bfd3ff295bf1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:28.717837 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c357bb-48c1-4417-8fe6-bfd3ff295bf1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:28.719027 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c357bb-48c1-4417-8fe6-bfd3ff295bf1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:28.719479 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c357bb-48c1-4417-8fe6-bfd3ff295bf1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:28.719894 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-75c357bb-48c1-4417-8fe6-bfd3ff295bf1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:28.862219 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-75c357bb-48c1-4417-8fe6-bfd3ff295bf1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:28.870234 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-75c357bb-48c1-4417-8fe6-bfd3ff295bf1 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 200 len: 1461 microversion: 2.1 time: 0.164387 Oct 20 09:11:28.871195 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 591/1181] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:28 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 1461 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:28.947402 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-58daf571-70c8-40fb-af7c-81e9ce82cde1 req-c3be5cda-c3dc-4373-889f-774c7cd6e504 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "842be7a4-f260-42e7-b34e-5f4369630d36", "name": "network-vif-deleted", "tag": "9d2af910-87ab-4f40-ac9b-a6a140aaca39"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:11:28.958497 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-58daf571-70c8-40fb-af7c-81e9ce82cde1 req-c3be5cda-c3dc-4373-889f-774c7cd6e504 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:28.959190 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-58daf571-70c8-40fb-af7c-81e9ce82cde1 req-c3be5cda-c3dc-4373-889f-774c7cd6e504 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:28.959627 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-58daf571-70c8-40fb-af7c-81e9ce82cde1 req-c3be5cda-c3dc-4373-889f-774c7cd6e504 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:28.972234 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-58daf571-70c8-40fb-af7c-81e9ce82cde1 req-c3be5cda-c3dc-4373-889f-774c7cd6e504 service nova] Creating event network-vif-deleted:9d2af910-87ab-4f40-ac9b-a6a140aaca39 for instance 842be7a4-f260-42e7-b34e-5f4369630d36 on hv-cinder-maste Oct 20 09:11:28.976997 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-58daf571-70c8-40fb-af7c-81e9ce82cde1 req-c3be5cda-c3dc-4373-889f-774c7cd6e504 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100403 Oct 20 09:11:28.977594 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 591/1182] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:11:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:11:29.893098 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:29.893701 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 842be7a4-f260-42e7-b34e-5f4369630d36] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:29.900119 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:29.900815 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:29.901640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:29.901920 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:29.902363 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:29.902811 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:29.917452 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] HTTP exception thrown: Instance 842be7a4-f260-42e7-b34e-5f4369630d36 could not be found. Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Request method failure captured: Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 HTTP/1.1 Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ7eeucTVD7DXQxu3n1HjFWasd0RlEnVQ7oXQ_5m8nyMDw2lV0skX3kJnuhAmasbAkMuuiafaZjUtOq0VyoEu1ww7D6-6g-C3iagvZtTiBjAkt4-LmcRo8IFuiY3H5D3qhIk_XxHFiP0oYVzcbk-uqb10V9y86uVAD2IxN7EFc4Z38E Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 98827dbef155480b9f26792b70014b4b Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:11:29.919116 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/98827dbef155480b9f26792b70014b4b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_98827dbef155480b9f26792b70014b4b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/98827dbef155480b9f26792b70014b4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/98827dbef155480b9f26792b70014b4b"}]}] Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 98827dbef155480b9f26792b70014b4b Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-TestVolumeBootPattern-1573908492-project Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 65ca2eae6db24bb094afaacdfb5c1980 Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-TestVolumeBootPattern-1573908492-project Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 842be7a4-f260-42e7-b34e-5f4369630d36 could not be found. Oct 20 09:11:29.921123 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '842be7a4-f260-42e7-b34e-5f4369630d36'} Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 842be7a4-f260-42e7-b34e-5f4369630d36 could not be found. Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:11:29.922745 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 842be7a4-f260-42e7-b34e-5f4369630d36 could not be found. Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 842be7a4-f260-42e7-b34e-5f4369630d36 could not be found. Oct 20 09:11:29.924128 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:11:29.925369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Returning 404 to user: Instance 842be7a4-f260-42e7-b34e-5f4369630d36 could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:11:29.925369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-963513c4-9ed3-44df-8fdf-92764d586a97 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36" status: 404 len: 111 microversion: 2.1 time: 0.031296 Oct 20 09:11:29.925369 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 592/1183] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:29 2022] GET /compute/v2.1/servers/842be7a4-f260-42e7-b34e-5f4369630d36 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:11:32.960391 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-220fa7c1-b366-42dd-b4cc-8855872228d0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:32.960983 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-220fa7c1-b366-42dd-b4cc-8855872228d0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:32.962006 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-64dd2cd1-a6bf-4493-b814-1c11f5cec025 req-86da4fd8-2c53-4fdc-8ce9-054ac32761d4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9c52d13f-7238-490c-a0cd-c1d2fff9afb0", "tag": "d5aa2d4a-df18-490f-b5c0-9074a308160e"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:11:32.967534 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-220fa7c1-b366-42dd-b4cc-8855872228d0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:32.968021 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-220fa7c1-b366-42dd-b4cc-8855872228d0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:32.968460 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-220fa7c1-b366-42dd-b4cc-8855872228d0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:32.969160 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-220fa7c1-b366-42dd-b4cc-8855872228d0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:32.969623 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-220fa7c1-b366-42dd-b4cc-8855872228d0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:32.970050 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-220fa7c1-b366-42dd-b4cc-8855872228d0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:32.973571 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-64dd2cd1-a6bf-4493-b814-1c11f5cec025 req-86da4fd8-2c53-4fdc-8ce9-054ac32761d4 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:32.974161 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-64dd2cd1-a6bf-4493-b814-1c11f5cec025 req-86da4fd8-2c53-4fdc-8ce9-054ac32761d4 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:32.974711 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-64dd2cd1-a6bf-4493-b814-1c11f5cec025 req-86da4fd8-2c53-4fdc-8ce9-054ac32761d4 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:32.986343 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-64dd2cd1-a6bf-4493-b814-1c11f5cec025 req-86da4fd8-2c53-4fdc-8ce9-054ac32761d4 service nova] Creating event network-changed:d5aa2d4a-df18-490f-b5c0-9074a308160e for instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 on hv-cinder-maste Oct 20 09:11:32.992227 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-64dd2cd1-a6bf-4493-b814-1c11f5cec025 req-86da4fd8-2c53-4fdc-8ce9-054ac32761d4 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034099 Oct 20 09:11:32.992824 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 593/1184] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:11:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:11:32.996230 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-220fa7c1-b366-42dd-b4cc-8855872228d0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:11:33.084605 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-220fa7c1-b366-42dd-b4cc-8855872228d0 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "DELETE /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 204 len: 0 microversion: 2.1 time: 0.128589 Oct 20 09:11:33.085375 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 592/1185] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:11:32 2022] DELETE /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:11:33.095510 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-8b1ac1eb-aec1-42eb-97d5-d27f581a396d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:33.096313 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-8b1ac1eb-aec1-42eb-97d5-d27f581a396d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:33.105769 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8b1ac1eb-aec1-42eb-97d5-d27f581a396d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:33.106362 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8b1ac1eb-aec1-42eb-97d5-d27f581a396d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:33.106884 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8b1ac1eb-aec1-42eb-97d5-d27f581a396d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:33.107677 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8b1ac1eb-aec1-42eb-97d5-d27f581a396d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:33.108179 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8b1ac1eb-aec1-42eb-97d5-d27f581a396d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:33.108664 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-8b1ac1eb-aec1-42eb-97d5-d27f581a396d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:33.242723 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-8b1ac1eb-aec1-42eb-97d5-d27f581a396d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:33.250950 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-8b1ac1eb-aec1-42eb-97d5-d27f581a396d tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1578 microversion: 2.1 time: 0.161319 Oct 20 09:11:33.251513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 594/1186] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:33 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1578 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:11:34.275599 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-a9615122-a2c8-452d-b973-cf1a0c2826a6 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:34.276302 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-a9615122-a2c8-452d-b973-cf1a0c2826a6 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:34.283335 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a9615122-a2c8-452d-b973-cf1a0c2826a6 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:34.283799 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a9615122-a2c8-452d-b973-cf1a0c2826a6 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:34.284238 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a9615122-a2c8-452d-b973-cf1a0c2826a6 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:34.284928 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a9615122-a2c8-452d-b973-cf1a0c2826a6 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:34.285379 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a9615122-a2c8-452d-b973-cf1a0c2826a6 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:34.285803 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-a9615122-a2c8-452d-b973-cf1a0c2826a6 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:34.419530 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-a9615122-a2c8-452d-b973-cf1a0c2826a6 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:34.427350 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-a9615122-a2c8-452d-b973-cf1a0c2826a6 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1578 microversion: 2.1 time: 0.156049 Oct 20 09:11:34.427917 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 593/1187] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:34 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1578 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:35.451450 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-fc2821cf-936b-4b6e-af56-b45e9122fda2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:35.452034 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-fc2821cf-936b-4b6e-af56-b45e9122fda2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:35.459924 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc2821cf-936b-4b6e-af56-b45e9122fda2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:35.460406 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc2821cf-936b-4b6e-af56-b45e9122fda2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:35.460839 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc2821cf-936b-4b6e-af56-b45e9122fda2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:35.461534 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc2821cf-936b-4b6e-af56-b45e9122fda2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:35.461984 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc2821cf-936b-4b6e-af56-b45e9122fda2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:35.462402 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-fc2821cf-936b-4b6e-af56-b45e9122fda2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:35.596428 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-fc2821cf-936b-4b6e-af56-b45e9122fda2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:35.604372 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-fc2821cf-936b-4b6e-af56-b45e9122fda2 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1578 microversion: 2.1 time: 0.157063 Oct 20 09:11:35.604946 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 595/1188] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:35 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1578 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:35.843353 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-2add2769-d3f8-47e7-a78e-64027f8ada2d req-8e699ceb-efd3-41fc-831c-d6853d469f32 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9c52d13f-7238-490c-a0cd-c1d2fff9afb0", "name": "network-vif-deleted", "tag": "d5aa2d4a-df18-490f-b5c0-9074a308160e"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:11:35.853254 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-2add2769-d3f8-47e7-a78e-64027f8ada2d req-8e699ceb-efd3-41fc-831c-d6853d469f32 service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:35.853732 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-2add2769-d3f8-47e7-a78e-64027f8ada2d req-8e699ceb-efd3-41fc-831c-d6853d469f32 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:35.854164 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-2add2769-d3f8-47e7-a78e-64027f8ada2d req-8e699ceb-efd3-41fc-831c-d6853d469f32 service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:35.865813 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-2add2769-d3f8-47e7-a78e-64027f8ada2d req-8e699ceb-efd3-41fc-831c-d6853d469f32 service nova] Creating event network-vif-deleted:d5aa2d4a-df18-490f-b5c0-9074a308160e for instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 on hv-cinder-maste Oct 20 09:11:35.870636 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-2add2769-d3f8-47e7-a78e-64027f8ada2d req-8e699ceb-efd3-41fc-831c-d6853d469f32 service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031336 Oct 20 09:11:35.871237 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 594/1189] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:11:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:11:36.636270 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:36.636835 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:36.643317 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:36.643788 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:36.644217 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:36.644892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:36.645333 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:36.645748 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:36.676892 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:36.677397 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:36.677950 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:36.678389 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lazy-loading 'fault' on Instance uuid 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:11:36.720363 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '65ca2eae6db24bb094afaacdfb5c1980', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98827dbef155480b9f26792b70014b4b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:11:36.727926 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-ab22ab16-2f69-4f0d-a82d-6410ea7fce6e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 200 len: 1230 microversion: 2.1 time: 0.095595 Oct 20 09:11:36.728489 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 596/1190] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:36 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 1230 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:11:37.752705 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:37.753317 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 9c52d13f-7238-490c-a0cd-c1d2fff9afb0] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:37.759379 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:37.759856 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:37.760289 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:37.760966 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:37.761413 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:37.761844 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:37.775278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] HTTP exception thrown: Instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 could not be found. Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Request method failure captured: Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 HTTP/1.1 Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ7eeucTVD7DXQxu3n1HjFWasd0RlEnVQ7oXQ_5m8nyMDw2lV0skX3kJnuhAmasbAkMuuiafaZjUtOq0VyoEu1ww7D6-6g-C3iagvZtTiBjAkt4-LmcRo8IFuiY3H5D3qhIk_XxHFiP0oYVzcbk-uqb10V9y86uVAD2IxN7EFc4Z38E Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 98827dbef155480b9f26792b70014b4b Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:11:37.776867 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/98827dbef155480b9f26792b70014b4b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_98827dbef155480b9f26792b70014b4b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/98827dbef155480b9f26792b70014b4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/98827dbef155480b9f26792b70014b4b"}]}] Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 98827dbef155480b9f26792b70014b4b Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-TestVolumeBootPattern-1573908492-project Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 65ca2eae6db24bb094afaacdfb5c1980 Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-TestVolumeBootPattern-1573908492-project Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 could not be found. Oct 20 09:11:37.778675 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '9c52d13f-7238-490c-a0cd-c1d2fff9afb0'} Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 could not be found. Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:11:37.779666 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 could not be found. Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 could not be found. Oct 20 09:11:37.781279 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:11:37.782779 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Returning 404 to user: Instance 9c52d13f-7238-490c-a0cd-c1d2fff9afb0 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:11:37.782779 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bb9cddb3-af50-47c6-8867-5f6b706c394f tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0" status: 404 len: 111 microversion: 2.1 time: 0.030937 Oct 20 09:11:37.782779 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 595/1191] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:37 2022] GET /compute/v2.1/servers/9c52d13f-7238-490c-a0cd-c1d2fff9afb0 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:11:38.205149 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:38.205671 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:38.211945 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:38.212420 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:38.212849 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:38.213529 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:38.213978 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:38.214400 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:38.228183 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.wsgi [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] HTTP exception thrown: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Request method failure captured: Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: request: DELETE /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee HTTP/1.1 Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept: application/json Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Accept-Encoding: identity Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Connection: close Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Length: 0 Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Content-Type: application/json Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Host: 10.222.0.36 Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: Openstack-System-Scope: None Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: User-Agent: python-urllib3/1.26.12 Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Auth-Token: gAAAAABjUQ7eeucTVD7DXQxu3n1HjFWasd0RlEnVQ7oXQ_5m8nyMDw2lV0skX3kJnuhAmasbAkMuuiafaZjUtOq0VyoEu1ww7D6-6g-C3iagvZtTiBjAkt4-LmcRo8IFuiY3H5D3qhIk_XxHFiP0oYVzcbk-uqb10V9y86uVAD2IxN7EFc4Z38E Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Id: None Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Domain-Name: None Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Identity-Status: Confirmed Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Is-Admin-Project: True Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Id: default Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Domain-Name: Default Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Id: 98827dbef155480b9f26792b70014b4b Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Project-Name: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Role: reader,member Oct 20 09:11:38.229831 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Roles: reader,member Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/98827dbef155480b9f26792b70014b4b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_98827dbef155480b9f26792b70014b4b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/98827dbef155480b9f26792b70014b4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/98827dbef155480b9f26792b70014b4b"}]}] Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Id: 98827dbef155480b9f26792b70014b4b Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-Tenant-Name: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User: tempest-TestVolumeBootPattern-1573908492-project Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Id: default Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Domain-Name: Default Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Id: 65ca2eae6db24bb094afaacdfb5c1980 Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: X-User-Name: tempest-TestVolumeBootPattern-1573908492-project Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: method: > Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: exception: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:11:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: action_args: {'id': '4b1d0b97-f53e-40dc-895f-48be8de838ee'} Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:11:38.232513 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:11:38.234640 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: ERROR nova.api.openstack.wsgi Oct 20 09:11:38.236629 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Returning 404 to user: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. {{(pid=119067) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:11:38.236629 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-98cf1d18-71d5-41f9-86f7-bbceb082975c tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "DELETE /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 404 len: 111 microversion: 2.1 time: 0.030892 Oct 20 09:11:38.236629 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 597/1192] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:11:38 2022] DELETE /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:11:38.240355 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:38.240993 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] [instance: 4b1d0b97-f53e-40dc-895f-48be8de838ee] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:11:38.246954 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:38.247429 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:38.247860 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:38.248543 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:11:38.248990 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:11:38.249932 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:11:38.263091 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] HTTP exception thrown: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Request method failure captured: Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee HTTP/1.1 Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ7eeucTVD7DXQxu3n1HjFWasd0RlEnVQ7oXQ_5m8nyMDw2lV0skX3kJnuhAmasbAkMuuiafaZjUtOq0VyoEu1ww7D6-6g-C3iagvZtTiBjAkt4-LmcRo8IFuiY3H5D3qhIk_XxHFiP0oYVzcbk-uqb10V9y86uVAD2IxN7EFc4Z38E Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 98827dbef155480b9f26792b70014b4b Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:11:38.264601 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/98827dbef155480b9f26792b70014b4b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_98827dbef155480b9f26792b70014b4b", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/98827dbef155480b9f26792b70014b4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/98827dbef155480b9f26792b70014b4b"}]}] Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 98827dbef155480b9f26792b70014b4b Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-TestVolumeBootPattern-1573908492 Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-TestVolumeBootPattern-1573908492-project Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 65ca2eae6db24bb094afaacdfb5c1980 Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-TestVolumeBootPattern-1573908492-project Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:11:38.266995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': '4b1d0b97-f53e-40dc-895f-48be8de838ee'} Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:11:38.270999 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. Oct 20 09:11:38.272613 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:11:38.274534 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Returning 404 to user: Instance 4b1d0b97-f53e-40dc-895f-48be8de838ee could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:11:38.274534 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-8b888135-5d6f-4242-b1ce-59f09de0d420 tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee" status: 404 len: 111 microversion: 2.1 time: 0.030010 Oct 20 09:11:38.274534 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 596/1193] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:11:38 2022] GET /compute/v2.1/servers/4b1d0b97-f53e-40dc-895f-48be8de838ee => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:11:39.556114 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-36907e59-ae05-426a-8ffb-ca02ab99892e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] Calling method '.version_select at 0x7f5afbaf7940>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:11:39.579486 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-36907e59-ae05-426a-8ffb-ca02ab99892e tempest-TestVolumeBootPattern-1573908492 tempest-TestVolumeBootPattern-1573908492-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1088348766" status: 202 len: 0 microversion: 2.1 time: 0.027145 Oct 20 09:11:39.580163 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 598/1194] 10.222.0.36 () {58 vars in 1263 bytes} [Thu Oct 20 09:11:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1088348766 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 20 09:12:22.816840 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-e06f3e64-8710-431d-9ad7-d18ac09d04c3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:12:22.817371 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e06f3e64-8710-431d-9ad7-d18ac09d04c3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:12:22.819455 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:12:22.819649 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_cell1:SELECT=40 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:12:22.820237 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG dbcounter [-] [119067] Writing DB stats nova_api:SELECT=23,nova_api:DELETE=1 {{(pid=119067) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:12:22.820482 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG dbcounter [-] [119068] Writing DB stats nova_cell1:SELECT=48,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=119068) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 09:12:22.824505 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [req-edd05500-6756-4768-b17e-259f6d5d4398 req-22b17e10-cb82-4db3-b862-7d79ad3e897c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c8a04d20-065b-47fd-b7e2-8238e34be187", "tag": "731aa0eb-33e7-4deb-b323-29e7908786e9"}]} {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:12:22.826410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e06f3e64-8710-431d-9ad7-d18ac09d04c3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:22.826927 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e06f3e64-8710-431d-9ad7-d18ac09d04c3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:22.827365 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e06f3e64-8710-431d-9ad7-d18ac09d04c3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:22.828064 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e06f3e64-8710-431d-9ad7-d18ac09d04c3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:22.828518 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e06f3e64-8710-431d-9ad7-d18ac09d04c3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:22.829585 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-e06f3e64-8710-431d-9ad7-d18ac09d04c3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:22.836520 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-edd05500-6756-4768-b17e-259f6d5d4398 req-22b17e10-cb82-4db3-b862-7d79ad3e897c service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:22.837059 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-edd05500-6756-4768-b17e-259f6d5d4398 req-22b17e10-cb82-4db3-b862-7d79ad3e897c service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:22.837453 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [req-edd05500-6756-4768-b17e-259f6d5d4398 req-22b17e10-cb82-4db3-b862-7d79ad3e897c service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:22.849274 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.compute.server_external_events [req-edd05500-6756-4768-b17e-259f6d5d4398 req-22b17e10-cb82-4db3-b862-7d79ad3e897c service nova] Creating event network-changed:731aa0eb-33e7-4deb-b323-29e7908786e9 for instance c8a04d20-065b-47fd-b7e2-8238e34be187 on hv-cinder-maste Oct 20 09:12:22.854530 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [req-edd05500-6756-4768-b17e-259f6d5d4398 req-22b17e10-cb82-4db3-b862-7d79ad3e897c service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036296 Oct 20 09:12:22.855442 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 599/1195] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:12:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 20 09:12:22.858784 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-e06f3e64-8710-431d-9ad7-d18ac09d04c3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Going to try to terminate instance {{(pid=119068) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 20 09:12:22.945304 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-e06f3e64-8710-431d-9ad7-d18ac09d04c3 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "DELETE /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 204 len: 0 microversion: 2.1 time: 0.132775 Oct 20 09:12:22.945995 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 597/1196] 10.222.0.36 () {58 vars in 1239 bytes} [Thu Oct 20 09:12:22 2022] DELETE /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 20 09:12:22.955176 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-92691200-12ff-4e43-99b9-36007cf6b90d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:12:22.955877 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-92691200-12ff-4e43-99b9-36007cf6b90d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:12:22.964780 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92691200-12ff-4e43-99b9-36007cf6b90d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:22.965835 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92691200-12ff-4e43-99b9-36007cf6b90d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:22.966967 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92691200-12ff-4e43-99b9-36007cf6b90d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:22.967869 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92691200-12ff-4e43-99b9-36007cf6b90d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:22.969072 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92691200-12ff-4e43-99b9-36007cf6b90d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:22.969908 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-92691200-12ff-4e43-99b9-36007cf6b90d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:23.103156 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-92691200-12ff-4e43-99b9-36007cf6b90d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:12:23.111404 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-92691200-12ff-4e43-99b9-36007cf6b90d tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.159989 Oct 20 09:12:23.111982 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 600/1197] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:12:22 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:12:24.135152 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-bbc85bd5-0c56-4f9e-932c-0655b912670a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:12:24.135848 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-bbc85bd5-0c56-4f9e-932c-0655b912670a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:12:24.142582 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbc85bd5-0c56-4f9e-932c-0655b912670a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:24.143002 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbc85bd5-0c56-4f9e-932c-0655b912670a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:24.143429 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbc85bd5-0c56-4f9e-932c-0655b912670a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:24.144130 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbc85bd5-0c56-4f9e-932c-0655b912670a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:24.144568 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbc85bd5-0c56-4f9e-932c-0655b912670a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:24.144981 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-bbc85bd5-0c56-4f9e-932c-0655b912670a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:24.284608 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.policy [None req-bbc85bd5-0c56-4f9e-932c-0655b912670a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119068) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:12:24.292355 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-bbc85bd5-0c56-4f9e-932c-0655b912670a tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.161775 Oct 20 09:12:24.292921 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 598/1198] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:12:24 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:12:25.315805 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-e06d8e71-0be6-458b-a04b-119e161edcd2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:12:25.316395 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-e06d8e71-0be6-458b-a04b-119e161edcd2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:12:25.322913 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e06d8e71-0be6-458b-a04b-119e161edcd2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:25.323392 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e06d8e71-0be6-458b-a04b-119e161edcd2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:25.323827 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e06d8e71-0be6-458b-a04b-119e161edcd2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:25.324535 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e06d8e71-0be6-458b-a04b-119e161edcd2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:25.324981 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e06d8e71-0be6-458b-a04b-119e161edcd2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:25.325401 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-e06d8e71-0be6-458b-a04b-119e161edcd2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:25.457907 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-e06d8e71-0be6-458b-a04b-119e161edcd2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:12:25.466033 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-e06d8e71-0be6-458b-a04b-119e161edcd2 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1565 microversion: 2.1 time: 0.154223 Oct 20 09:12:25.466624 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 601/1199] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:12:25 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1565 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 20 09:12:25.684143 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [req-d6545890-40f0-47c9-b8f1-7cb35b235ce6 req-947c80ef-ce73-48e1-af1f-2029efdf892f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c8a04d20-065b-47fd-b7e2-8238e34be187", "name": "network-vif-deleted", "tag": "731aa0eb-33e7-4deb-b323-29e7908786e9"}]} {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 20 09:12:25.694781 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-d6545890-40f0-47c9-b8f1-7cb35b235ce6 req-947c80ef-ce73-48e1-af1f-2029efdf892f service nova] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:25.695259 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-d6545890-40f0-47c9-b8f1-7cb35b235ce6 req-947c80ef-ce73-48e1-af1f-2029efdf892f service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:25.695686 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [req-d6545890-40f0-47c9-b8f1-7cb35b235ce6 req-947c80ef-ce73-48e1-af1f-2029efdf892f service nova] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:25.707827 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.compute.server_external_events [req-d6545890-40f0-47c9-b8f1-7cb35b235ce6 req-947c80ef-ce73-48e1-af1f-2029efdf892f service nova] Creating event network-vif-deleted:731aa0eb-33e7-4deb-b323-29e7908786e9 for instance c8a04d20-065b-47fd-b7e2-8238e34be187 on hv-cinder-maste Oct 20 09:12:25.712624 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [req-d6545890-40f0-47c9-b8f1-7cb35b235ce6 req-947c80ef-ce73-48e1-af1f-2029efdf892f service nova] 10.222.0.36 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032201 Oct 20 09:12:25.713278 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 599/1200] 10.222.0.36 () {64 vars in 1317 bytes} [Thu Oct 20 09:12:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 20 09:12:26.488754 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:12:26.489356 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.compute.api [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119067) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:12:26.496345 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:26.496845 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:26.497441 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:26.498296 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:26.498811 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:26.499236 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:26.531751 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:26.532866 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:26.533295 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG oslo_concurrency.lockutils [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119067) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:26.533740 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.objects.instance [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lazy-loading 'fault' on Instance uuid c8a04d20-065b-47fd-b7e2-8238e34be187 {{(pid=119067) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 20 09:12:26.577528 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.policy [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '21311bc5eef742f5a9eb81462a701ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d1505e085847329f7480c9bc08f1d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119067) authorize /opt/stack/nova/nova/policy.py:199}} Oct 20 09:12:26.586394 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-907f7ca9-09e6-42dc-aad4-cbfebeb33981 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 200 len: 1224 microversion: 2.1 time: 0.101477 Oct 20 09:12:26.586979 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 602/1201] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:12:26 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 1224 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 20 09:12:27.611683 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '>' {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:12:27.612379 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.compute.api [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] [instance: c8a04d20-065b-47fd-b7e2-8238e34be187] Fetching instance by UUID {{(pid=119068) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 20 09:12:27.618520 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:27.619034 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:27.619477 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:27.620164 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Acquiring lock "dd273305-b4a6-40de-9d39-cde54b55a76f" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 20 09:12:27.620622 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 20 09:12:27.621070 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG oslo_concurrency.lockutils [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Lock "dd273305-b4a6-40de-9d39-cde54b55a76f" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119068) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 20 09:12:27.634215 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.wsgi [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] HTTP exception thrown: Instance c8a04d20-065b-47fd-b7e2-8238e34be187 could not be found. Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Request method failure captured: Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: request: GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 HTTP/1.1 Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept: application/json Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Accept-Encoding: identity Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Connection: close Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Length: 0 Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Content-Type: application/json Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Host: 10.222.0.36 Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: Openstack-System-Scope: None Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: User-Agent: python-urllib3/1.26.12 Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Auth-Token: gAAAAABjUQ_P7wULiGe6MfOEdK8twCHZqnycCGRqJ2cBJuW5OXk_EIU9eOICkaP0SVO16cC_3oTgn7T0GOkvft6qTrf-cZXOgldhB9MqccB1zYzC677cJFHLvyK_w3SiZMg1_g7zaZk-jVf--SbeCrNCikwsu4bRa34PvAioR1my1xr_bI_qAfQ Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Id: None Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Domain-Name: None Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Identity-Status: Confirmed Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Is-Admin-Project: True Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Id: default Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Domain-Name: Default Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Id: 45d1505e085847329f7480c9bc08f1d8 Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Project-Name: tempest-TestShelveInstance-2073296570 Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Role: reader,member Oct 20 09:12:27.635854 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Roles: reader,member Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/45d1505e085847329f7480c9bc08f1d8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36:8080/v1/AUTH_45d1505e085847329f7480c9bc08f1d8", "adminURL": "http://10.222.0.36:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/compute/v2/45d1505e085847329f7480c9bc08f1d8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.36/volume/v3/45d1505e085847329f7480c9bc08f1d8"}]}] Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant: tempest-TestShelveInstance-2073296570 Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Id: 45d1505e085847329f7480c9bc08f1d8 Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-Tenant-Name: tempest-TestShelveInstance-2073296570 Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User: tempest-TestShelveInstance-2073296570-project Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Id: default Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Domain-Name: Default Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Id: 21311bc5eef742f5a9eb81462a701ceb Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: X-User-Name: tempest-TestShelveInstance-2073296570-project Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: method: > Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: exception: Instance c8a04d20-065b-47fd-b7e2-8238e34be187 could not be found. Oct 20 09:12:27.637673 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: action_args: {'id': 'c8a04d20-065b-47fd-b7e2-8238e34be187'} Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: {{(pid=119068) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c8a04d20-065b-47fd-b7e2-8238e34be187 could not be found. Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 20 09:12:27.638679 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c8a04d20-065b-47fd-b7e2-8238e34be187 could not be found. Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c8a04d20-065b-47fd-b7e2-8238e34be187 could not be found. Oct 20 09:12:27.640410 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: ERROR nova.api.openstack.wsgi Oct 20 09:12:27.642063 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: DEBUG nova.api.openstack.wsgi [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Returning 404 to user: Instance c8a04d20-065b-47fd-b7e2-8238e34be187 could not be found. {{(pid=119068) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 20 09:12:27.642063 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: INFO nova.api.openstack.requestlog [None req-eb1d3460-23e2-415e-b2c6-d7bb05664dc4 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187" status: 404 len: 111 microversion: 2.1 time: 0.030998 Oct 20 09:12:27.642063 dv-cinder-master-3-iscsi devstack@n-api.service[119068]: [pid: 119068|app: 0|req: 600/1202] 10.222.0.36 () {58 vars in 1236 bytes} [Thu Oct 20 09:12:27 2022] GET /compute/v2.1/servers/c8a04d20-065b-47fd-b7e2-8238e34be187 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 20 09:12:27.793217 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: DEBUG nova.api.openstack.wsgi [None req-32ab5265-f25a-451c-b199-9f3dbab4f2c7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] Calling method '.version_select at 0x7f5afb937160>' {{(pid=119067) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 20 09:12:27.816608 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: INFO nova.api.openstack.requestlog [None req-32ab5265-f25a-451c-b199-9f3dbab4f2c7 tempest-TestShelveInstance-2073296570 tempest-TestShelveInstance-2073296570-project] 10.222.0.36 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1651321010" status: 202 len: 0 microversion: 2.1 time: 0.027170 Oct 20 09:12:27.817199 dv-cinder-master-3-iscsi devstack@n-api.service[119067]: [pid: 119067|app: 0|req: 603/1203] 10.222.0.36 () {58 vars in 1254 bytes} [Thu Oct 20 09:12:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1651321010 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)